How to import Stations Open Data provided by the French governement

The French government, using its open data portal https://www.data.gouv.fr, is providing a monthly updated list of stations. The dedicated page is here:

And the last update is here:

https://www.data.gouv.fr/fr/datasets/r/50625621-18bd-43cb-8fde-6b8c24bdabb3

Do you have a standard file import format? Maybe it would be possible to transform this file from the French government in a data format that suits you?

1 Like

Thanks, it looks like this also has an API so we can find the latest version. I’ll investigate to see if I can build an import for this as it has an open data license.

So based on the first draft of this import, this data has 10282 items, of which 5986 will be skipped as duplicates (either in the data or because we appear to have them already), 4296 POIs will be added.

Hello Christopher,
I’ve requested recently to add the operator Sigief and become to implement by hand the some CCS station around me.

I’ve realized that the information are already in the file from the source you already imported it.

Could you import the latest version of this file, all the SIGEIF is inside as per today.
https://www.data.gouv.fr/fr/datasets/r/8d9398ae-3037-48b2-be19-412c24561fbb
last update : 13 mai 2022

Thanks

I’ll look into what’s required, some imports are difficult to repeat if we have keyed on an ID that later changes.

From my understanding you have already imported a general set of french data organized IRVE, so if dataset are structure in the same way, it should be straightforward.
The hint should be in the tag (metadonnées) “irve” :

If you confirm me that IRVE structure is ok, and block on some french data structured not according to IRVE, let me know, I’ll convert them for you. (Especially the other set that I’ve found)
Likely you’ll not be able to have a script to pull them monthly, but neither the stations will grow significantly by months :slight_smile:
Once you use the dataset let me know I’ll define openchargemap as a data user on the dataset, closing the opensource loop :slight_smile:

So digging more in the data set, there is one main dataset irve, that you already imported in 03/20, and multiple small one that are not included… like the Sigief, Tesla, ionity etc…
If you confirm that the IRVE datable format is still ok, I can concatenate all the dataset relevant from this website and provide you one file.

what we need is a single API call to exort all the data, previoulsy we used this https://www.data.gouv.fr/api/1/datasets/?q=5448d3e0c751df01f85d0572&page=0&page_size=20 which doesn’t work anymore, so we’d need to track down the right API call. The export must include all network operators, not one at a time.

I had actually given up on most imports like this because the data is fairly mixed quality and in some parts is wrongly aggregated. Instead we want networks to agree to provide us data in a standard format directly: https://openchargemap.org/site/about/datasharing - currently only one network has offered to do that and is yet to provide us the necessary API credentials to work on that.

I appreciate your offer of concatenating data etc but that’s not a repeatable/sustainable way of working.

Thanks for your feedback,

I agree with you on all the points, however I would like to add :

  • In my area, significant amount of stations are missing (mainly the one that are not included in the file that you importer 2 years ago). A one time import, like you already did would help significantly to address these missing networks.
  • This french server allow to use an API to download files (doc in french :Documentation de API catalogue des données ouvertes - data.gouv.fr - api.gouv.fr)
    As you mentioned each dataset version has a unique ID, that make your old link obsolete. Not the best for recurrent data pulling. However I’m confident, that with a script (curl based), the latest dataset version ID could be retrieve, thus the data linked to this ID. (It’s about procession json payload)
    My capabilities are only in python/bash, I could try to generate a script pulling all the relevant french data and concatening them in one relevant dataframe.

So through my previous messy msg there is 2 different objectives :

  • One time import, wider than the previous one, to compensate most of the missing network in France.
  • Regular update, monthly or 6 month base to keep the database update.

These data are quite valuable, generated by the network owner shared in a opensource database, I’m not aware of any country centralizing such data in one place (however several files).

Even a one shot import, as you did 2 years ago, is really interesting as new chargers are not build every morning :wink:

Ps: Could you clarify what does mean " is wrongly aggregated." ?
Non maching dataframe structure ?

Thanks, here is the code for our existing import from this provider, if we can find the latest dataset ID then I can try that: ocm-system/ImportProvider_DataGouvFr.cs at master · openchargemap/ocm-system · GitHub

Other countries (in particular the UK) do have other open data sets but they all suffer from inconsistent storage of connector, equipment information, badly formatted addresses and incorrect latitude/longitude information (sometime swapped so points end up in the sea etc). While it’s technically possible to do more to refine and work around these problems it’s not something I’m going to spend time on, someone else is welcome to try though.

Thanks Christopher,

I’ve started to fork a python script from the french gov to generate a consolidate file, I’ll tune it for our need and data cleanup.
Will see what I’ll get.