# 2019 Data exploration, minerals data feedback

**URL:** <https://forum.goatech.org/t/2019-data-exploration-minerals-data-feedback/928>\
**Category:** BI Lab Network\
**Tags:** xrf, minerals\
**Created:** [October 9, 2020, 7:56pm UTC](https://forum.goatech.org/t/2019-data-exploration-minerals-data-feedback/928 "2020-10-09T19:56:38Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![gbathree](https://forum.goatech.org/user_avatar/forum.goatech.org/gbathree/32/3_2.png) [@gbathree](https://forum.goatech.org/u/gbathree)\
**Post date:** [October 9, 2020, 7:56pm UTC](https://forum.goatech.org/t/2019-data-exploration-minerals-data-feedback/928/1 "2020-10-09T19:56:38Z")

</div>

@jill @DanT Dan Kittredge, Emily, Mariette --\>

This is to supply a cleaned and simplified version of the data for evaluating minerals data, especially to identify outliers. This requires seeing the final data w/ some metadata, like crop type, and links to the original XRF files to check for screwed up runs or interfering peaks.

Then, the **best way to identify outliers** is to access the **2019-data-simplified-for-minerals.csv** below, search for outlier values in the XRF columns, find the associated sample\_id, and identify that sample\_id in the **Data 2019** referenced below. The sample\_id can be found as in the file name. First, I suggest organizing the data by crop type, as outliers are easier to spot by crop.

- **Access to Files:** [https://drive.google.com/drive/folders/1KVDAgVXWXDxVOYRGBdgXc\_Zg3-\_kqH1U?usp=sharing](https://drive.google.com/drive/folders/1KVDAgVXWXDxVOYRGBdgXc_Zg3-_kqH1U?usp=sharing)
  - **2019-data-simplified-for-minerals.csv** - Integrated, cleaned file for reviewing XRF Data. Added some headers to organize data better, removed some unnecessary columns. This can be used to identify outliers.
  - **Data 2019** link - link to raw XRF Files. Use the ‘soil, helium’ and ‘food, helium’ folders for soil and food, respectively.
  - **RFC 2019 - XRF Data** - Calibrated outputs from the raw XRF files

Here is some data tables Octavio produced to generate more easily interpretable results:

- [minerals data table](https://octavioduarte.gitlab.io/public-briefs/soilMineralRelations)
- [crops data table](https://octavioduarte.gitlab.io/public-briefs/mineralRelations)

**Precision** - Finally, some analysis was done in 2018 on the impact of grind size with carrots and spinach. We ran triplicate fine, coarse, and rough grind sizes on the same carrot and the same set of spinach leaves. In the **RFC 2019 - XRF Data** above go to the ‘Grind Size Test Results’ tab to see the results. We should probably re-run this kind of analysis this winter to update the data, but at least for now here’s what we’ve got.

In addition, **here are other ways to check out the 2019 data** , which is still in process but getting close --\>

**2019 Data**

- **Download csv (easy to read version)** - [go here](https://app.our-sci.net/#/dashboard/by-dashboard-id/rfc-2019-data-partner-results), click to download “All Data - human readable names”
- **Download csv (full version… 910 columns, beware!)** - [go here](https://app.our-sci.net/#/dashboard/by-dashboard-id/rfc-2019-data-partner-results), scroll to bottom, click to download “All Data - complete”
- **Data explorer** (not yet for prime time, but feel free to play): [https://farmersfeedback.herokuapp.com/pages/inner](https://farmersfeedback.herokuapp.com/pages/inner)?
- @DanT FILL IN WITH FINAL TABLES FROM OCTAVIO
