Skip to content
Snippets Groups Projects
Commit ae0db8bd authored by Benoit Favre's avatar Benoit Favre
Browse files

update code highlighting

parent 3254328d
No related branches found
No related tags found
No related merge requests found
...@@ -10,8 +10,9 @@ Install requirements ...@@ -10,8 +10,9 @@ Install requirements
The REST services require bottle.py to be installed. The REST services require bottle.py to be installed.
pip install --user -r requirements.txt -U ```
pip install --user -r requirements.txt -U
```
REST services REST services
------------- -------------
...@@ -23,35 +24,47 @@ server of your choice (the default WSGIRefServer is quite slow, you may install ...@@ -23,35 +24,47 @@ server of your choice (the default WSGIRefServer is quite slow, you may install
For instance, you can run the rest/test.py script which reverses character strings. For instance, you can run the rest/test.py script which reverses character strings.
rest/test.py ```
rest/test.py
```
It replies to three URLs: / is a short textual help for using the service. It replies to three URLs: / is a short textual help for using the service.
curl http://localhost:1234 ```
curl http://localhost:1234
```
It also recognizes GET requests of the form /test/<text>, where <text> is the It also recognizes GET requests of the form /test/<text>, where <text> is the
string you want to reverse. string you want to reverse.
curl http://localhost:1234/test/hello ```
curl http://localhost:1234/test/hello
```
Finally, for large inputs, it recognizes POST queries where the input string is Finally, for large inputs, it recognizes POST queries where the input string is
specified in a "text" form parameter. specified in a "text" form parameter.
curl --form "text=hello" http://localhost:1234/test ```
curl --form "text=hello" http://localhost:1234/test
```
A generic REST service is also available. It runs a custom command and feeds it A generic REST service is also available. It runs a custom command and feeds it
with inputs through stdin and collects output from stdout. The following with inputs through stdin and collects output from stdout. The following
example echoes the inputs as output. The name parameter set the name of the example echoes the inputs as output. The name parameter set the name of the
service in the url. service in the url.
rest/generic.py --port 1234 --name "cat" --command "cat" ```
curl http://localhost:1234/cat/hello rest/generic.py --port 1234 --name "cat" --command "cat"
curl http://localhost:1234/cat/hello
```
The command can also be made persistent (run in the background) and fed line by The command can also be made persistent (run in the background) and fed line by
line through stdin, and its output read line by line from stdout. Note that for line through stdin, and its output read line by line from stdout. Note that for
this to work, the command needs to flush stdout after each input. this to work, the command needs to flush stdout after each input.
rest/generic.py --port 1234 --name --command "awk '{x+=1;print x,\$0;fflush()}'" --persistent True ```
rest/generic.py --port 1234 --name --command "awk '{x+=1;print x,\$0;fflush()}'" --persistent True
```
WARNING: the generic REST service does not enforce any kind of security. WARNING: the generic REST service does not enforce any kind of security.
...@@ -66,7 +79,9 @@ Repository-integrated services poll the repository for new documents, process ...@@ -66,7 +79,9 @@ Repository-integrated services poll the repository for new documents, process
them and push back annotation sets. In order to get access to the repository, them and push back annotation sets. In order to get access to the repository,
you can create a tunnel with (given a proper ssh key is setup): you can create a tunnel with (given a proper ssh key is setup):
util/repository.py --tunnel ```
util/repository.py --tunnel
```
The util/repository.py script has a lot more commands available. It can also be The util/repository.py script has a lot more commands available. It can also be
used as a python module as full-functional repository client. used as a python module as full-functional repository client.
...@@ -75,7 +90,9 @@ Once the tunnel is setup, you can try the test annotator which polls the ...@@ -75,7 +90,9 @@ Once the tunnel is setup, you can try the test annotator which polls the
repository and adds phony annotations. That script allows to choose the host and repository and adds phony annotations. That script allows to choose the host and
port of the repository. port of the repository.
repo/test.py ```
repo/test.py
```
This script gets all documents which don't have the AMU_hasTest feature, puts This script gets all documents which don't have the AMU_hasTest feature, puts
generated annotations in the AMU_Test annotation set. Then, it sets the generated annotations in the AMU_Test annotation set. Then, it sets the
...@@ -84,7 +101,9 @@ AMU_hasTest feature to True. ...@@ -84,7 +101,9 @@ AMU_hasTest feature to True.
After a few documents are processed, you can kill the script and run the After a few documents are processed, you can kill the script and run the
cleaner which deletes all annotation sets created by the previous script. cleaner which deletes all annotation sets created by the previous script.
repo/clean.py --presence_feature AMU_hasTest --annotation_set AMU_Test ```
repo/clean.py --presence_feature AMU_hasTest --annotation_set AMU_Test
```
The first way to integrate a novel processing module to the repository is to The first way to integrate a novel processing module to the repository is to
use the generic annotator. It grabs all documents which don't have a given use the generic annotator. It grabs all documents which don't have a given
...@@ -98,9 +117,11 @@ its expectations and contain the right fields. Overwise, an error is returned. ...@@ -98,9 +117,11 @@ its expectations and contain the right fields. Overwise, an error is returned.
For example, you can write a script which computes the length of the json For example, you can write a script which computes the length of the json
representation of a document, and creates a new "checksum" annotation with it. representation of a document, and creates a new "checksum" annotation with it.
```
cat script.sh cat script.sh
awk '{print "[{\"type\": \"checksum\", \"features\": {\"value\":"length()"}, \"start\": 0, \"end\": 0}]"}' awk '{print "[{\"type\": \"checksum\", \"features\": {\"value\":"length()"}, \"start\": 0, \"end\": 0}]"}'
repo/generic.py --command ./script.sh --mark_feature "AMU_hasChecksum" --annotation "AMU_Checksum" repo/generic.py --command ./script.sh --mark_feature "AMU_hasChecksum" --annotation "AMU_Checksum"
```
The command can be run once for each document, or just once in the background. The command can be run once for each document, or just once in the background.
When doing so, the command is fed line by line and its output is read line by When doing so, the command is fed line by line and its output is read line by
...@@ -111,6 +132,7 @@ The second way to integrate a novel processing module to the repository is to ...@@ -111,6 +132,7 @@ The second way to integrate a novel processing module to the repository is to
subclass the repository.AnnotationGenerator class in python. See the subclass the repository.AnnotationGenerator class in python. See the
repo/test.py script for an example. repo/test.py script for an example.
```python
class Annotator(repository.AnnotationGenerator): class Annotator(repository.AnnotationGenerator):
def __init__(self): def __init__(self):
query = '_MISSING_=MarkingFeature&_MAX_=20' query = '_MISSING_=MarkingFeature&_MAX_=20'
...@@ -122,4 +144,4 @@ class Annotator(repository.AnnotationGenerator): ...@@ -122,4 +144,4 @@ class Annotator(repository.AnnotationGenerator):
... # generate annotation ... # generate annotation
client.put_annotation_set(doc_id, 'AnnotatinoName', ...) client.put_annotation_set(doc_id, 'AnnotatinoName', ...)
client.put_features(doc_id, {'MarkingFeature': True}) client.put_features(doc_id, {'MarkingFeature': True})
```
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment