Skip to content
Snippets Groups Projects
Commit 821ec273 authored by Alexis Nasr's avatar Alexis Nasr
Browse files

added some rules in french tokenizer

parent 980dd8bd
Branches master
No related tags found
No related merge requests found
...@@ -15,9 +15,13 @@ extern int defait_amalgames; ...@@ -15,9 +15,13 @@ extern int defait_amalgames;
[0-9]+,[0-9]* printf("%s", yytext); [0-9]+,[0-9]* printf("%s", yytext);
[ \t]+ printf("\n"); [ \t]+ printf("\n");
\. printf("\n."); [ ]*\. printf("\n.");
\, printf("\n,"); [ ]*\? printf("\n?");
… printf("\n…"); [ ]*\! printf("\n!");
[ ]*, printf("\n,");
[ ]*: printf("\n:");
[ ]*; printf("\n;");
[ ]*… printf("\n…");
' printf("'\n"); ' printf("'\n");
’ printf("'\n"); ’ printf("'\n");
-je printf("\n-je"); -je printf("\n-je");
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment