The most simple crawler¶
You can define the most simple crawler job by writing a
~/.fscrawler/test/_settings.yaml file as follow:
This will scan every 15 minutes all documents available in
dir and will index them into
test_doc index. It will connect to an
elasticsearch cluster running on
name is a mandatory field.