Retirement of Data Imports etc

OCM
1

Since the beginning of OCM we have imported some other open data sources such as the US AFDC etc until open charge map as a way of capturing this data without users having to enter it manually. Imports make up about 49% of our current data.

I am planning to discontinue most data imports and instead rely on user edits from now on. The main reason for this is that I personally don’t want to maintain the data import system in my free time and I do not want to develop any more data imports. I would consider refining one OCPI format data import for general purpose use but would instead prefer that networks who wish to push data to us do so via our UI/app or via our API (this necessitates that any updates must not overwrite community edits made since the providers last update).

The help of the community in refining our data is much appreciated but I think it’s worth flagging that I’m the only person in the world currently maintaining the software for OCM and I’m not generally interested in developing improvements/fixes for OCM. I do force myself to work on it sometimes but I rarely use it myself and any fixes/support I do for it are just to try to keep it going. I would like other community members to make themselves know if they are interesting in working on/supporting any aspects of the system or if they have ideas they would like to implement.

1 Like
2

Hi Christopher, first I’d like to thank you for all the work.
As for data imports using OCPI one advantage I see would be if it could be possible to show the online status of the stations (plugs occupied, free, out-of-service). I know this is asking a lot and would require a lot of programming.
For importing new stations I agree with your proposal.

I would like to help, but my programming skills are quite limited, I think I could be more useful checking the data quality and managing edits in my local area.

1 Like
3

Hi Alex, thanks - I’ve just spread a little thin on other work and OCM isn’t really at the top of my list of things to do.

I actually tweeted about this yesterday (https://twitter.com/webprofusion/status/1382523925020647424?s=20) and got a lot of positive interest in helping maintain OCM but most people who responded don’t really know much about it. It’s difficult to find people with the right skills (or who can attain them) who will also donate free time to quite a large topic. Currently, I’m the only person running this, and my interest in the project varies daily depending on what other priorities I have, sometimes I dread opening the OCM community site in case someone is asking me to do something, which doesn’t seem like a scalable or healthy situation.

Perhaps rather than getting rid of imports we’ll (as a community) eventually change how they are done but I see more value long term in crowdsourcing the data - that does mean we don’t get equipment status updates. Status is transient information and we should really hold that separately from the core data because it changes so frequently.

For user contributed status information, in english speaking regions I’d argue that PlugShare is by far the dominant way for people to check for charging locations and to check-in. Our check-in numbers are pretty low and our check-in UI (in our app) is not particularly refined. I’m not sure if that’s something we should be pursuing further or not.

Our niche is really providing data for other apps/service to use (e.g. A Better Route Planner etc) and this is also a potential method for revenue generation as high-volume API users could pay a subscription (and we could in turn automatically expand the performance of our API servers).

Just ideas currently, but we need to start to think about how to operate when there are 1000 times as many charging stations and 1.4 billion EVs (i.e. all cars).

1 Like
4

Hi Christopher,
I am on the point in my webapp development that I want to switch to using ( providing auto-check in will come later ) your database. The API , when correctly used should be able to auto add feedbacks like succesfull charged on plugtype, kWh charged per POI ( using feedback via OBDII dongle from the car )
When you have OCPI connection ready, let me know if I can help parties here in the Netherlands to start using them.
Kind Regards,
Edwin Glaser
Founder Nissan Electric Club

5

Hi Christopher. First of all, like @AlexMol said, thanks for all the work you put into this. I read your tweets and can see where you’re coming from.

I haven’t put in much effort to using the open charge map API so far, beyond investigation and fit for purpose assessment. I have a lot of plans on what I want to do with it, but like you, other work has got in the way, so I’m looking at probably July before I’m free to do anything.

I have lots of questions :grin: …

  • This sounds like you’re the only one supporting and developing this service?
  • What are the differences between Open Charge Map and PlugShare? Am I correct in saying PlugShare is a “pay for API key” service only? How does the data volume for Open Charge Map compare with PlugShare?
  • What are your thoughts on a paid for service and a basic limited free service? Or are you effectively becoming another PlugShare?
  • How about establishing a foundation? Is there possibility for donation from bigger companies in the EV world. (I’d balk on saying investment - that’s a headache you probably don’t want)
  • Do you leverage the fact this is not for profit - free resources here and there?
  • Would you be open to rewriting the .Net code so you could at least move away from Windows, licensing etc.?
  • How much effort is there in integrating with OCPI? Have you had a lot of demand for OCPI?
  • Does the app bring in much revenue, or at all? Would it be worth pausing that to get some breathing space?
  • Is there benefit on open sourcing the entire codebase?
  • What would it take for you to be able to work on this full time?

I’d have to think about what time I could provide to help out - but also why I’ve asked the questions above. (btw, I ultimately don’t care what stack you’re using). If you’d rather answer any of these more privately feel free to contact me directly.

Thanks.

6

Thanks @Primodi these are great questions.

This sounds like you’re the only one supporting and developing this service?

Yes, I am the only person developing the software/tools/imports/app and organising the website/community etc. I do it as a hobby, less so nowadays. The actual data is refined by the community of users themselves, I don’t do any of that (or try not to). This discussion forum here is the only interaction I have with the community, so if there is any other organizing going on elsewhere, I don’t know about it.

What are the differences between Open Charge Map and PlugShare? Am I correct in saying PlugShare is a “pay for API key” service only?

Yes, plugshare requires a commercial agreement to use their data (or rather, that data users have submitted). OCM makes the data available for free under a combination of open data licenses (imports have other licenses, so there’s a mix). Our API is currently free for everyone, some people abuse this terribly, other’s are good about it.

How does the data volume for Open Charge Map compare with PlugShare?

I think we have less than 10% of the active direct (app.wesbites) userbase that PlugShare has, possibly even 5%. We have probably about 80% of the charging locations (or more). Hard to tell. We have a high number of API consumers spread across a few hundred different apps/services.

What are your thoughts on a paid for service and a basic limited free service? Or are you effectively becoming another PlugShare?

When talking about our API, that’s a service where we query our data and return the data to the app/service that’s requesting it. That in turn costs us money to do. User can just clone the data from github though

How about establishing a foundation? Is there possibility for donation from bigger companies in the EV world. (I’d balk on saying investment - that’s a headache you probably don’t want)

I’d be open to it, I just don’t want to have to organise it. It may be personality type but I’d rather stick pins in my eyes than ‘jump on a call’ all the time.

Do you leverage the fact this is not for profit - free resources here and there?

We have had some help, MapBox in particular gave us a period of free credit on their map tile platform after Google Maps got very expensive. Currently though we have nothing from anyone.

Would you be open to rewriting the .Net code so you could at least move away from Windows, licensing etc.?

Over the long term I don’t really care what technology is used but despite how it might appear on the surface we do have a lot of code and a lot of technology. Any developer can make an API, database etc in a weekend, rewriting all of OCM would be a minimum 4-month project, if done properly (new unit testing, CI/CD for everything, re-engineering and data migrations, QA/testing ) then closer to 6-8 months.

Our current costs are about 25% Windows and the rest is Linux based. Our API/Website runs on .net core (.net 5) with MongoDB as a caching layer, which is all cross platform. Our actual data store is SQL Server Express (free) which we try to minimise and is technically available on Linux. Porting to MySQL was tried a few years ago but I didn’t have the reason/motivation to really see that through to production. .Net was originally chosen because it was the technology I used for my commercial work, so it was my strongest skill.

How much effort is there in integrating with OCPI? Have you had a lot of demand for OCPI?

We have a very slight gap in our data model which would ideally be addressed (grouping of connectors into logical EVSE) but overall taking an OCPI data feed is a few days works, maybe a week or so to get it to production quality, so not super massive but still substantial. Exporting is OCPI roughly the same. The broader issue is how/when to perform de-duplication (crowdsourced vs imported locations etc) and data licensing.

Does the app bring in much revenue, or at all? Would it be worth pausing that to get some breathing space?

The app itself is simply a means for end user to consume and contribute to the data, it is not intended to be a serious competitor to commercial apps like PlugShare. Our hope has always been that many competing apps would appear and to some extent that has happened in some areas but they usually don’t feed any data back in. We/I technically could produce an app which closely matches any of the other EV apps, that has not been a goal and it’s not time I personally wanted to spend and I think it de-incentivizes other apps to build their products.

We had an Open Collective for about a year and that had one regular contributor who was donating $20 per month, that was great in itself but to make something even a part time endeavor you really need to pay someone’s salary, and that person has to wear all the hats for the entire organisation. To run as a multiple person organisation would run into hundreds of thousands of dollars per year, so that’s not likely without a sudden and mysterious benefactor.

Is there benefit on open sourcing the entire codebase?

The entire codebase is already open sourced. We have had very few code contributions. Often this is because people don’t know the technology but realistically at the level we’re pitching it we don’t need people who struggle to code, we (generally) need full-on career software developers which usually means people who can code in multiple languages, or show are prepared to learn anything required.

What would it take for you to be able to work on this full time?

I already have a moderately successful software business (selling https://certifytheweb.com) which ironically grew out of the need to manage multiple SSL certificates for OCM which I didn’t want to have to pay for, so I think I’m beyond the point of myself working on OCM full time. Possibly part time (or someone else full time) and possibly via putting a subscription/fees on the OCM API. I’d budget $100K USD per annum for one staff developer with a few years experience (salary is less but all payroll has tax overheads). This could happen via my company (Webprofusion Pty Ltd) but only if the project was funded.

We did briefly offer companies a hosted API mirror solution (for $199 per month) but this was both too expensive for those that said they wanted it and too cheap to make it a viable business (because you’d need lots of subscribing companies to pay a single salary, at least around 45-50 subscribers at that rate).

7

I have previously intimated elsewhere (maybe twitter?) that I’d be happy to talk to established EV related organizations about operating the Open Charge Map project/brand, they could then use whatever technology they like as long as it provides open data.

This would perhaps be the best thing for the project overall as a pre-built organization (with other revenue streams) would have developers, marketing, lawyers, social media and money. Interestingly PlugShare (or rather it’s parent company) has changed hands a few times and last time I looked it was owned by a European power company.

8

Hi Edwin,

Our API has supported comments/checking for a long time so you should be able to use that, although our documentation could do with some work. What we didn’t really have was the ability for a service to post as a specific OCM user without directly using that users credentials. We do have some implementation for facility to some extent but it may be incomplete and nobody is using it. That’s an example of one of the countless ‘little’ things that we need to improve upon.

You can however let a user login to the OCM API, then post checkins as that user. Our OCM app uses our API, so anything it can do your app could also do.

We don’t have checkin metadata on plug type or kw but you could include it as part of the checkin comment.

1 Like
9

That seems right.
There are 2 things that I would improve, that would help a lot of users on other services:

  1. Opening hours. Currently there is no standardized way of inputing opening hours on charge points. ABRP may route you, during the night, through a charging point that is closed. Something like the OpenStreetMap opening_hours tagging scheme could be useful.
  2. Prices, a text box is ok if the prices are complicated, but in many cases you could get away with having 3 fields for
    currency per activation
    currency per minute
    currency per kWh
    This would help other services figure out the price of charging.

I believe OCPI has fields for prices, and as the fluctuate a lot it might be better suited for a separate layer (like the one with in-use plugs), but I don’t know if OCPI includes opening hours.

1 Like
10

I’d be happy to talk to established EV related organizations about operating the Open Charge Map project/brand, they could then use whatever technology they like as long as it provides open data.

I know I’ve mentioned this before, but OpenStreetMap would be an ideal candidate for this. I know there are some issues with conflicting licenses, etc. on the imported ones, but even a partial merge of data is going to be beneficial.

OpenChargeMap can act as a user friendly front end for adding stations and viewing them - with OpenStreetMap being the data repository.

I’m at the point where when I find new stations I add it to Plugshare (horrid, but they have the audience) and OpenStreetMap, I sometimes get around to OpenChargeMap but as I become more time-poor I think it will be OCM that falls off priority list.

Of course you’ve done a fantastic job to run this site @Christopher - I don’t want to denigrate that - just considering what’s best for the future of the site given your limited personal resources.

11

It’s still a possibility and it has been mentioned before but unless someone official (i.e. from the OSM foundation, not volunteers with an interest in OSM) coordinates it I don’t think it will happen because it would need real commitment on all sides. We can’t import our data into OSM unless they want it and the charging station data in OSM itself varies hugely in quality (there is little to zero chance we will import from OSM). OSM is also a key/value metadata store so there is a challenge as to how to maintain and extend fidelity of data as well (equipment level info) and tackle lack of standardised metadata values etc (e.g. operator ids, connector ids). If making that leap you’d also want something that maps well to OCPI for data interchange.

I’m not against it, but I’m not going to organise it :slight_smile:

12

@Christopher have you had any luck finding someone to take on the technical elements of this project yet? I’m not necessarily saying that I’m volunteering, but I have the technical skills and the interest to take on some work.

13

No, we’ve had no new volunteers for that job. Feel free have a go (from open data sources) - I think in your case reviewing how the UK NCR import works would be most relevant.

14

In my current ( alpha version ) app I collect/ calculate also the distance you need to travel extra on your route when you want to use a qc . 3 fields with :distance , uom and direction of travel would be great to give back to the database.

15

Oh … since there can be multiple approach directions possible i would like to be able to post at least to 4 or max 12 directions. To keep track of delta distance to be travelled

16

Thanks, you should maintain your own database for this derived information.

1 Like