Data Interoperability Working Group

I know it’s short notice, but we’re doing a quick impromptu call today to discuss some technical issues around data interoperability at 2pm EST. If you can join, great! If not, no worries we’ll take some notes.

Here’s the Zoom call info:

Join from PC, Mac, Linux, iOS or Android: https://zoom.us/j/216973907

Or iPhone one-tap :
US: +16699006833,216973907# or +16465588656,216973907#
Or Telephone:
Dial(for higher quality, dial a number based on your current location):
US: +1 669 900 6833 or +1 646 558 8656
Meeting ID: 216 973 907
International numbers available: https://zoom.us/u/cOzm63dcn

Just a great blog about why use JSON-LD

Will try call in for 2pm but why the impromptu, I’m definitely going to miss the beginning :frowning:

1 Like

The call is because @MarsBenzle asked a question on Riot so we’re just opening up the call we were going to have anyway to anyone who wants to join

Aaron, Jane and I spoke a moment ago on the perceived technical problems with interoperability in open source agriculture. I wanted to gather this information because I will speak with a researcher here at MSU (Michael O’Rourke) for a potential webinar on developing better standards for collaboration among open source ag software developers and researchers (http://toolbox-project.org/). I will post the notes from that discussion in the webinar page. Please, feel free to add ideas on technical problems that you have experienced. I find explicit examples to be especially useful.

Technical issues with interoperability

-In general developers solve for their own individual use case or a researcher is collecting data they want to collect in a format they would prefer. (CSV or proprietary format)
-Researchers don’t have much context for larger software product.
-Most developers are working with client server so that client and server can talk to each other but not for a other groups.
-Data formats are poorly specified
-People use a specific programming language if they want to share it they have go back and look at the format they used and relayed the language.(software is not developed with the expectation of collaboration)
-In software development people are trying to solve their own problem there is not a structured way of describing things.
Current solutions

  • W3C ontologies for people to have shared data models, describes the schema being used,
    -Ontologies are listed to give a common language to talk about certain topics, common vocabularies, when they are sharing data they can use those vocabularies.
    -People can ignore the work on ontologies and coordinate among each other to achieve interoperability.
    Problem with current solutions
    -Big disconnect between developers, researchers and the tools they would need to use to use the ontologies.
    -People are not aware of the tools available, and there use has a whole other education step to understand how to use them.
    -There are only a few people working on ontologies and schemas so they are not all that developed.
    -Format of the tools to use ontologies are not all that accessible to most developers. You would really only know of them if you are into open data.
    -Even software developers don’t know about the existence of ontologies. Not a lot of aggregation of those tools.
1 Like

Maybe this group will do half of the work for us?
http://godan.info/working-groups/publication-and-alignment-authoritative-vocabularies-food-working-group

Wish I could find a mailing list/other public forum to engage with them. Seems to have launched Nov 2017

1 Like

So I emailed the RDA Agricultural Data Interest Group (IGAD) mailing list and got a pretty comprehensive response from someone on that project. for those interested in ontologies for food this sounds really promising.

The full response from Valeria Pesce is publicly visible here including their proposal docs but in brief:

"aims at discussing possible alignments or crosswalks between the most authoritative classifications related to food and nutrition…

The final objective of this work is to provide a set of semantically interlinked URIs for key food product concepts as an infrastructural component that will facilitate the development of both software for actors in the food value chain and added-value integrated services that need to track food products."

1 Like

Dorn started this Google Doc to be shared here: https://docs.google.com/document/d/16fzhspMpNj00996iV1wD0PIOrP9UGng864CHVKB-DEM/edit

1 Like

Jeff Herrick from LandPKS also pointed out this in relation to metadata standards:

It seems that the ecology community has a stronger consensus (EML – Ecological Metadata Language).

Agricultural Metadata Element Set (AgMES) | AIMS

They are interested in joining this discussion as well.

We will identify one person to serve as lead on this in the next few months.

EML info at https://knb.ecoinformatics.org/#external//emlparser/docs/index.html

My understanding is that it is used by most of the big networks including LTER and NEON (which also uses a number of ISO standards), and well as a number of other (especially NSF-funded) projects.

However for ag, a more relevant effort is being led by the National Ag Library and the LTAR network. They are starting with ISO 19115: Geographic information - Metadata standard, and I believe that Jeff Campbell at NAL is working on developing tools based (initially) on that standards.

1 Like

Looks like we’re converging around 11am or 1pm on Wed May 23rd in the doodle poll. If you haven’t done so please respond now

@weston @ravibhattarai pinging you guys because I know you expressed interest in this working group but haven’t seen you in the thread or riot.

Growstuff has a reasonable database of crops which can be queried by JSON

You can get all the data they have about plantings of tomatoes for example. Or a human readable tomato page

The nice thing about it is that it’s all linked back to the Wikipedia page for that crop too.

2 Likes

I just replied to doodle.

I spoke with Micheal O’Rourke from MSU yesterday on possibly working with his group on developing a shared set of standards around interoperability or possibly open source tech in general. His group usually works with scientists doing collaborative, interdisciplinary, or trans-disciplinary work (working closely with stakeholders) on finding what that group’s shared goals and practice should be. Here is a link to the documents he sent me to give more information on what they do https://drive.google.com/open?id=1Kjq3VbnsIiSPWC6eaauNnBQVdD-mC8TE :
(1) Their fee schedule
(2) A one-page description of them
(3) An older paper that goes into some detail about what they do — this was written for an audience of scientists
(4) A couple of Toolbox instruments to give you a sense of the kind of things groups talk about

They run it with an initial dialog session with 12-15 people then do a larger virtual workshop with the whole group (3 hours). The fee is kind of steep, Michael seemed kind of open to haggling but we would have to see about that. It seems like a good opportunity to work out some of the larger cultural and ethical issues open source struggles with but it is really up to you all. Let me know what you all think.

2 Likes

Jut my 2c but while I agree facilitation is incredibly valuable the harder part would be getting the right people to attend such a event. GOAT is definitely not the only group discussing Ag data interoperability, I think we’ll have more luck joining/building with some other groups efforts already out there and contributing unique use cases/building demo uses - eg say apply some of their work to FarmOS?

:?

1 Like

I think Jane’s $0.02 is good to keep in mind. Along those lines, as the work on prepping the API for GIPS as a service, I’m going to ensure that it uses semantics that are congruent with whatever we ontology we converge on…be it a GOAT original, GODAN, or OATS. It is also precisely the route I’d like to take for any GIPS+farmOS integration.

++ Ian

Unless there are strong objections, I’m going to go ahead and close the Doodle poll, which would make our first working group meeting Wed, May 23rd at 1pm EST (10am PST/19:00 CET). Out of the nine people that responded, there is one person who cannot make that time. Unfortunately, I don’t think that we’re going to get any better consensus with a group of this size…

I propose that we record the meeting and make it plus the notes available for anyone who cannot make it. Does that work for everyone?

Works for me! Thanks @aaronc!

Works for me @aaronc! Ta!

@aaronc

Sounds good anyone have desire to host else I’ll make a zoom link?

Wed 23 May 1pm EST

Do you have a paid account? If so go ahead and host! Otherwise I have one