Task Queue Service: queued (tentacle) tasks (preys), while a task is a URL to fetch
850
Kraken helps you to fetch a lot of URLs.
Hello I am KRAKEN - URLs are my prey:
/status
GET: get the status of this kraken
/tentacle/<name>
PUT / POST : create or overwrite a new tentacle with body {"bandwidth": <int>, "retry": <int>}
PATCH : patch the tentacle change it bandwidth and number of retries with body {"bandwidth": <int>, "retry": <int>}
GET : get the status of an existing tentacle
DELETE : get rid of the tentacle
/tentacle/<name>/<preyId>
PUT/POST : let me catch some prey with body { "url" : <string>, "priority" : <int>, ["locks" : [<string>, ...], "method" : <string>, "body" : <string>, "tags" : [<string>, ...]] }
With "locks" you can define resource names. Kraken will try to lock these resources before running a prey. This helps you to prevent running preys with the same resource(s) in parallel.
Some curl examples for locale development. Please consider NOT to run kraken in insecure mode on production systems!
start kraken
./kraken-linux -address "127.0.0.1:8080" -insecure -config "example-config.yaml"
create tentacle
curl -k -H "Content-Type: application/json" -X PUT -d '{"retry":3, "bandwidth":2}' 127.0.0.1:8080/tentacle/foo
add a prey
curl -k -H "Content-Type: application/json" -X PUT -d '{ "url" : "https://www.google.com", "priority" : 100}' 127.0.0.1:8080/tentacle/foo/prey1
get status of a tentacle
curl -k 127.0.0.1:8080/tentacle/foo
delete tentacle
curl -k -X DELETE 127.0.0.1:8080/tentacle/foo
docker run --rm -it -v $PWD/example-config.yaml:/etc/kraken/config.yaml -p="8765:80" foomo/kraken:latest
Content type
Image
Digest
Size
2.3 MB
Last updated
about 10 years ago
docker pull foomo/kraken