2019 Data exploration, minerals data feedback

@jill @DanT Dan Kittredge, Emily, Mariette -->

This is to supply a cleaned and simplified version of the data for evaluating minerals data, especially to identify outliers. This requires seeing the final data w/ some metadata, like crop type, and links to the original XRF files to check for screwed up runs or interfering peaks.

Then, the best way to identify outliers is to access the 2019-data-simplified-for-minerals.csv below, search for outlier values in the XRF columns, find the associated sample_id, and identify that sample_id in the Data 2019 referenced below. The sample_id can be found as in the file name. First, I suggest organizing the data by crop type, as outliers are easier to spot by crop.

  • Access to Files: https://drive.google.com/drive/folders/1KVDAgVXWXDxVOYRGBdgXc_Zg3-_kqH1U?usp=sharing
    • 2019-data-simplified-for-minerals.csv - Integrated, cleaned file for reviewing XRF Data. Added some headers to organize data better, removed some unnecessary columns. This can be used to identify outliers.
    • Data 2019 link - link to raw XRF Files. Use the ‘soil, helium’ and ‘food, helium’ folders for soil and food, respectively.
    • RFC 2019 - XRF Data - Calibrated outputs from the raw XRF files

Here is some data tables Octavio produced to generate more easily interpretable results:

Precision - Finally, some analysis was done in 2018 on the impact of grind size with carrots and spinach. We ran triplicate fine, coarse, and rough grind sizes on the same carrot and the same set of spinach leaves. In the RFC 2019 - XRF Data above go to the ‘Grind Size Test Results’ tab to see the results. We should probably re-run this kind of analysis this winter to update the data, but at least for now here’s what we’ve got.

In addition, here are other ways to check out the 2019 data, which is still in process but getting close -->

2019 Data

  • Download csv (easy to read version) - go here, click to download “All Data - human readable names”
  • Download csv (full version… 910 columns, beware!) - go here, scroll to bottom, click to download “All Data - complete”
  • Data explorer (not yet for prime time, but feel free to play): https://farmersfeedback.herokuapp.com/pages/inner?