Development¶

The Parser class¶

The Parser class defines the core parsing logic and serves as a base class for all concrete parsers, which must inherit from it.

Available attributes¶

Execution flow diagram¶

Configurable built-in parsers¶

Existing parsers¶

Several default parsers are available in Geotrek. They are more or less generic scripts designed to save time when creating data mappings. Below is the list of these scripts for each touristic information system:

* Geometry is only available without dynamic segmentation. It is not possible to import geometries for treks in a database using dynamic segmentation.

APIDAE¶

Apidae is a collaborative network and a tourism information management platform. It enables tourist offices, local authorities, service providers, and private partners to share, structure, and distribute tourism data (accommodations, events, sites, services, etc.). It serves as a common reference system at the local, regional, and national levels.

Configure APIDAE access¶

To access your APIDAE data, you must define the following attributes in your parser class (inherited from an APIDAE base parser):

• api_key: Your personal API key provided by APIDAE

• project_id: The ID of your APIDAE project

• selection_id: The ID of the selection containing the data to import

These values are required and should be filled in according to your APIDAE (formerly SITRA) configuration.

class HebergementParser(TouristicContentApidaeParser):
    label = "Hébergements"
    api_key = 'xxxxxxxx'
    project_id = 9999
    selection_id = 99999
    category = "Hébergement"
    type1 = ["Camping"]
    type2 = ["3 étoiles", "Tourisme et Handicap"]  # just remove this line if no type2

class ImportTreksApidae(ApidaeTrekParser):
    label = "Import trek with eid"
    label_fr = "Import itinéraires avec identifiant externe"
    label_en = "Import trek with eid"
    api_key = 'xxxxxxxx'
    project_id = 9999
    selection_id = 99999
    eid = 'eid'
    practices_mapped_with_activities_ids = {
        'Pratique Pédestre': [
            3172,  # Itinéraire de randonnée pédestre
        ],
    }
    practices_mapped_with_default_activities_ids = {
        'Pratique Pédestre': 3184,  # Sports pédestres
    }

class DrinkingWaterPoint(ApidaeServiceParser):
    label = "Drinking water points"
    provider = "Apidae"
    selection_id = 12345
    service_type = "Drinking water point"

class PicNicTable(ApidaeInfrastructureParser):
    label = "Picnic tables"
    provider = "Apidae"
    selection_id = 12345
    infrastructure_type = "Picnic table"

Activate translations¶

expand_translations is an option that can be activated for a specific field in field_options. It automatically fills the translated versions of the field using multilingual data from APIDAE.

Example:

field_options = {
    "name": {
        "expand_translations": True,
        "required": True
    }
}

Tourinsoft¶

Tourinsoft is a Tourism Information System developed by the company Ingénie for tourism organizations in France, such as Departmental Tourism Committees (CDT), Tourism Development Agencies (ADT), and Tourist Offices. This system allows for the centralization, management, and standardized dissemination of tourism-related information.

class RestaurationParser(TouristicContentTourinsoftParser):
    """Restauration parsers"""
    label = "Restauration"
    category = "Restauration"
    url = "<Touristic content data feed URL"  # In the form https://api-v3.tourinsoft.com/api/syndications/decibelles-data.tourinsoft.com/<id>?format=json"

Cirkwi¶

Cirkwi is a platform for distributing tourism content (treks, points of interest, digital guides) aimed at tourism professionals. It helps promote tourism data through websites, mobile apps, or interactive kiosks using widgets or APIs, relying on a library of shared or proprietary content.

class ImportTreksCirkwi(CirkwiTrekParser):
    url = "<Treks data feed URL>"  # In the form https://ws.cirkwi.com/flux/<user>/<code>/circuits.php?widget-id=<id>
    user = "<Username>"
    password = "<Password>"
    auth = (user, password)
    label = "Cirkwi's treks"
    delete = True
    create = True
    provider = "Cirkwi"

class ImportTouristicContentCirkwi(CirkwiTouristicContentParser):
    url = "<Treks data feed URL>"  # In the form https://ws.cirkwi.com/flux/<user>/<code>/circuits.php?widget-id=<id>"
    user = "<Username>"
    password = "<Password>"
    auth = (user, password)
    label = "Cirkwi's touristic content"
    delete = True
    create = True
    provider = "Cirkwi"
    # results_path = "circuit/pois/poi"  # Uncomment this line if the touristic content to be imported come from the same feed as  treks

LEI¶

The LEI (Lieu d’Échanges et d’Informations) was the former shared tourism information system used in Alsace to centralize and distribute regional tourism data (accommodations, events, sites, etc.).

class XXXLEIEventParser(LEITouristicEventParser):
    label = "LEI TouristicEvent"
    url = "https://url.asp"

class XXXLEIEventParser(LEITouristicEventParser):
    label = "LEI TouristicEvent"
    url = "https://url.asp"

Marque Esprit Parc¶

The Esprit Parc brand promotes tourist offers committed to the preservation of nature and local know-how in national park areas.

class XXXEspritParcParser(EspritParcParser):
    label = "Marque Esprit Parc"
    url = "https://gestion.espritparcnational.com/ws/?f=getProduitsSelonParc&codeParc=XXX"

class MielEspritParcParser(EspritParcParser):
    label = "Miel Esprit Parc national"
    url = "https://gestion.espritparcnational.com/ws/?f=getProduitsSelonParc&codeParc=XXX&typologie=API"
    constant_fields = {
        'category': "GeotrekCategoryName",
        'published': True,
        'approved': True,
        'deleted': False,
    }
    m2m_constant_fields = {
        'type1': "GeotrekTypeName",
    }

OpenStreetMap¶

OpenStreetMap (OSM) is a collaborative, open-source mapping database that provides freely accessible geographic data, maintained by a global community of contributors. OpenStreetMap parsers retrieve OSM data using the Overpass API.

Basic configuration of OSM parsers¶

class MaisonDuParcParser(InformationDeskOpenStreetMapParser):
    provider = "OpenStreetMap"
    tags = [{"amenity": "ranger_station"}]
    default_fields_values = {"name": "Maison du Parc"}
    type = "Maisons du parc"

class RestaurantParser(OpenStreetMapTouristicContentParser):
    provider = "OpenStreetMap"
    tags = [{"amenity": "restaurant"}]
    default_fields_values = {"name": "restaurant"}
    category = "Restaurants"
    type1 = "Restaurant"

class HistoryParser(OpenStreetMapPOIParser):
    provider = "OpenStreetMap"
    tags = [
        {"historic": "yes"},
        {"historic": "castel"},
        {"historic": "memorial"},
        {"historic": "fort"},
        {"historic": "bunker"},
        {"building": "chapel"},
        {"building": "bunker"},
    ]
    default_fields_values = {"name": "Historic spot"}
    type = "Histoire"

class DistrictParser(OpenStreetMapDistrictParser):
    provider = "OpenStreetMap"
    tags = [
        [{"boundary": "administrative"}, {"admin_level": "6"}], # departement
        [{"boundary": "administrative"}, {"admin_level": "4"}], # region
    ]
    default_fields_values = {"name": "district"}

class RegionalNatureParkParser(OpenStreetMapDistrictParser):
    provider = "OpenStreetMap"
    tags = [{"protection_title"="parc naturel régional"}]
    default_fields_values = {"name": "parc naturel régional"}
    area_type = "Inconnu"

class DirectionalParser(OpenStreetMapSignageParser):
    provider = "OpenStreetMap"
    tags = [{"information": "guidepost"}]
    default_fields_values = {"name": "guidepost"}
    type = "Directionelle"

class TableParser(OpenStreetMapInfrastructureParser):
    provider = "OpenStreetMap"
    tags = [
        {"leisure": "picnic_table"},
        {"tourism": "picnic_table"}
    ]
    default_fields_values = {"name": "picnic table"}
    type = "Table"

class ClimbingSiteParser(OpenStreetMapOutdoorSiteParser):
    provider = "OpenStreetMap"
    tags = [{"sports": "climbing"}]
    default_fields_values = {"name": "climbing site"}
    practice = "Escalade"

class CityParser(OpenStreetMapCityParser):
    provider = "OpenStreetMap"
    tags = [
        [{"boundary": "administrative"}, {"admin_level": "8"}],
    ]
    default_fields_values = {"name": "city"}
    code_tag = "ref:INSEE"

Query configuration¶

By default, the parser uses the German Overpass server: https://overpass-api.de/api/interpreter/.

You can override this by setting a custom URL in the url attribute of the OpenStreetMapParser class.

Overpass queries are written in Overpass QL. Query configuration is handled through the query_settings attribute, which includes:

• bbox_margin (default: 0.0): A proportional buffer applied to the query bounding box. It expands the area by a fraction of its width to ensure surrounding features are included. (exemple: if bbox_margin is 0.05 then the bbox will be expanded by 5%)

• osm_element_type (default: nwr): Specifies the types of elements to retrieve: "node", "way", "relation", or "nwr" (all three).

• output (default: "geom"): Specifies the data returned by the Overpass API.

  • geom: return the object type, the object ID, the tags and the geometry

  • tags: return the object type, the object ID and the tags

The tags attribute defines the set of tag filters to be used with the Overpass API (see MapFeatures to get a list of existing tags). It is a list where each element is either:

• A dictionary: representing a single tag filter (e.g., {"highway": "bus_stop"})

• A list of dictionaries: representing a logical AND across all contained tags (e.g., [{« boundary »: « administrative »}, {« admin_level »: « 4 »}] means the object must have both tags).

The Overpass query will return the UNION of all top-level items.

For example:

self.tags = [
    [{"boundary": "administrative"}, {"admin_level": "4"}],
    {"highway": "bus_stop"}
]

means: return objects that either have both boundary=administrative AND admin_level=4, OR have highway=bus_stop.

All the objects parsed by the OpenStreetMap parsers will be those contained in the settings.SPATIAL_EXTENT bounding box. You can change the bounding box by overriding get_bbox_str().

Handling translated fields¶

OpenStreetMap supports multilingual fields using tags like name:fr, following the ISO 639-1 standard.

During import, the parser maps translated fields (e.g., name, description) based on the model and the languages defined in settings.MODELTRANSLATION_LANGUAGES. For each language, it creates a mapping such as name_fr → name:fr.

For the default language (settings.MODELTRANSLATION_DEFAULT_LANGUAGE), a special mapping is applied: it includes a fallback to the base tag (e.g., name) and maps it to the base Geotrek field name (e.g., name). This allows for filtering operations without relying directly on the default language code.

If a translation is missing, the field remains unset unless a fallback value is provided in default_fields_values using the pattern {field}_{lang}.

When no translation exists for the default language, the base OpenStreetMap tag (e.g., name) is used. This can lead to incorrect language display if the OSM default does not match the Geotrek instance’s default language.

Translation logic can be customized in custom parsers by overriding the translation_fields method.

Pièce-jointes¶

OpenStreetMapParser automatically attaches files from wikimedia_commons and image tags found in the data. A CC BY-SA 4.0 license is assigned to each imported file, as specified by the OpenStreetMap license.

For more information on how attachments work, consult this section.

Import from multiple sources¶

When importing data for the same model using two (or more) different sources, the provider field should be used to differenciate between sources, allowing to enable object deletion with delete = True without causing the last parser to delete objects created by preceeding parsers.

In the following example, Provider_1Parser and Provider_2Parser will each import their objects, set the provider field on these objects, and only delete objects that disappeared from their respective source since last parsing.

class Provider_1Parser(XXXXParser):
    delete = True
    provider = "provider_1"

class Provider_2Parser(XXXParser):
    delete = True
    provider = "provider_2"

The following example would cause NoProviderParser to delete objects from Provider_2Parser and Provider_1Parser.

class Provider_1Parser(XXXXParser):
    delete = True
    provider = "provider_1"

class Provider_2Parser(XXXParser):
    delete = True
    provider = "provider_2"

class NoProviderParser(XXXParser):
    delete = True
    provider = None # (default)

Further information¶

Overriding base parsers methods¶

Base parser classes and built-in configurable parsers implement parsing logic that is either generic or specific to certain data sources. You can override these methods in your custom parsers to adapt behavior to your needs.

However, this should only be done when necessary, as custom implementations will prevent your parsers from benefitting from future improvements or bug fixes in the base methods.

Detailed operation of attachment parsing¶

AttachmentParserMixin lets a parser link (and optionally download) media files to any object it imports (signage, infrastructures, POIs, touristic content, events, etc). The mixin is located in geotrek/common/parsers.py and must be inherited by your parser:

class ExampleParser(AttachmentParserMixin, Parser):

    # Parser configuration …

Attributes¶

The following attributes can be customized:

• download_attachments (default: True): Whether to download and store attachments via Paperclip. If set to False, attachments are only linked. Requires PAPERCLIP_ENABLE_LINK = True in Django settings.

• base_url (default: ""): Base URL prepended to each relative attachment path returned by filter_attachments.

• delete_attachments (default: True): After the new attachments have been processed, every existing attachment that is *not* present in the current feed (or whose file has been replaced) is permanently removed.

• filetype_name (default: "Photographie"): Label of the FileType model assigned to all imported files. If it does not exist in the database, the import will fail with a warning:

  FileType '<name>' does not exist in Geotrek-Admin. Please add it
  

• non_fields (default: {"attachments": _("Attachments")}): Maps the internal attachments field to the field name(s) containing attachments data in the external source.

• default_license_label (default: None): If specified, this license will be assigned to all imported attachments. If the license does not exist, it will be created automatically.

Filtering attachments¶

The filter_attachments method formats the external source data to match with the internal format.

If the attachment data has a different structure than the default filter_attachments, the method must be overridden.

See the geotrek/common/parsers.py file to see more about attachments.

Geometry filtering in Geotrek parsers¶

In some cases, you may want to restrict imported objects to a specific geographic area already defined in geotrek model instance (ex: a City or District). This can be done by defined the parser’s intersection_geom attribute

This attribute is a dictionary with the following keys:

• model: The Django model containing the reference geometry object.

• app_label: The Django application where the model is defined.

• geom_field: The name of the geometry field in the model.

• object_filter: A dictionary to identify the reference object (e.g., using an ID).

The object_filter must return exactly one object:

• If no object is found, the parser raises a blocking error.

• If multiple objects are returned, only the first will be used, which may cause unexpected behavior.

Conditional deletion with delete = True¶

If delete attribut is set to True, the parser will automatically delete existing objects of the current model that do not intersect the reference geometry.

Linking source objects via eid¶

Starting with Geotrek-admin 2.117.0, the eid (external ID) field displayed on the object detail page can now include a clickable link to the original source object.

To enable this, a new database model called Provider has been added. This model can be managed through the Django admin interface and includes the following fields:

• Name

• Link template (an HTML snippet used to build the link using the eid)

• Copyright

The link template should contain the {{object.eid}} placeholder, which will be replaced by the actual external ID. For example:

<a href="https://example.com/objects/{{object.eid}}" target="_blank">{{object.eid}}</a>

Fixtures are available for two providers: OpenStreetMap and Apidae. These predefined Provider records can be loaded during a new installation. See the fixture documentation to see more about fixtures.