Trouble posting sensor data to FarmOS from Arduino/ESP8266

#1

Hi everyone! I’m new here so I might as well introduce myself and my project for context.

My name is Connor, I’m a rising senior pursuing a B.S. in Environmental Studies and I’m working on a project on an organic vineyard for my senior thesis this summer which involves a network of IoT soil-moisture sensors. For my sensor prototype I’m using an Arduino Uno, an ESP8266-01, the SoilWatch 10 by PinoTech, and a battery power controller based on the information in this blog (https://alanbmitchell.wordpress.com/2011/10/09/design-of-the-extended-battery-life-power-controller-for-the-arduino/) using an Adafruit PowerBoost 500 and a PIC12F683 microcontroller. I’ve become familiar with FarmOS over the past year and want to use it as the repository for my data, but I’m having trouble posting the data to my sensor’s URL via the ESP8266.

I’m very new to all of this and so my naivete will likely reflect in my code and/or approach to the task. The immediate issue is that I am trying to use an AT+CIPSEND command to post my sensor data to FarmOS as per the code in the link below. I can’t get the ESP8266 to return the “>” character which is supposed to preclude the actual POST request. All other AT commands work with my set up as of right now.

If anyone could help me out with this I’d really appreciate it–I’ve been trying to work through it for several days now based on what I can find elsewhere online with no luck so far. I know there are likely many alternatives to my approach. If you have experience with a better method using Arduino/ESP8266, I’d be interested to hear!

#2

Sounds like an awesome project - unfortunately I can’t help with the code trouble, but am interested to hear more about your project! I’m also measuring soil moisture in my research (I’m a graduate student at the University of Vermont) and am wondering how you feel about the SoilWatch 10 by PinoTech, and how many of them are you using in the field? Also, where is the organic vineyard you are monitoring? I hope someone can help with the code & FarmOS integration :slight_smile:

#3

Hey @barbieri, thanks for the message! My project intends to focus on the interaction between cover crops and soil across a slope to see how different crop arrangements effect the way soil retains moisture in response to rain events for the summer. I’m going to be using a stratified random arrangement of cover crop treatments on either side of three vine rows along the same slope which will include annual ryegrass, crimson clover, ryegrass + clover, the polyculture of grasses and forbs already growing between my sample vines, and a bareground control. I’m hoping to use these sensors to take a moisture reading from each node every 15 minutes until the fall or even grape harvest in October/November, which I will compare with precipitation data and data related to soil characteristics (texture, OM, etc.).

I like the SoilWatch 10 so far! What drew me to it was the cable length (1.5 m) and its weatherproofing. All of my tests with it have worked well so far and the customer support from PinoTech has been good. I intend to use 45-48 for my project at Green Bird Cellars in northern Michigan (Leelanau peninsula). I’d love to hear more about your research! I’m sure there may be somewhere you’ve written about it before on the forums that I can find.

#4

Hi @connor! I’ve never tried pushing from ESP8266 specifically, but I posted a link to this in the FIoT chat (https://riot.im/app/#/room/#fiot:matrix.org) - I know @donblair and @chrowe have experimented with it.

If I remember correctly, there were some difficulties in posting to HTTPS (SSL encrypted URLs) from ESP8266 - but it was possible to do it from an ESP32 instead. I’m not sure if there was ever a solution for the 8266 on that issue.

Also, you may want to edit your code sample above to remove the public/private keys - otherwise anyone who sees that can post to those URLs. If you want, you can edit your sensor configuration in farmOS to change those keys (just change a single letter/number and it will be fine) - then obfuscate your code example (or just replace with “[public-key]” and “[private-key]”).

One other thing: when using https, it will be port 443, not port 80 - although the HTTPS issue itself is the bigger issue I think.

#5

Mike has proven he has a good memory!

https://gitlab.com/fiot/HUZZAH-FarmOS where we posted our code.

I ended up using Node-RED as middleware to solve the https problem. See https://gitlab.com/fiot/HUZZAH-FarmOS/blob/master/arduino/HUZZAH-NodeRED/HUZZAH-NodeRED.ino for that code.

Once you are in Node-RED you can use the http request function node to send the data to FarmOS. See https://youtu.be/rCl06YBb4cM?t=1345 for a video where I demonstrate how that works.

@donblair was working on the upython/esp32 approach which I think he got working.

#6

Thanks for the links Chris!

Mike has proven he has a good memory!

Ha! I don’t know about that… I’m pretty good at remembering specific coding problems/solutions (or even where in the code something is), but ask me what I ate for dinner yesterday and I’d have a hard time. There are a lot of kinds of “memory” (aka ways of indexing) and I think I’ve specialized in some at the expense of others. :wink:

Reminds me of this: https://www.wired.com/2011/12/london-taxi-driver-memory/

+1 on the Node Red approach! It requires a second device in addition to the ESP8266, but if you think of that as the “gateway” then it makes it easy to scale out to more sensors in the future. And it separates concerns - so the ESP8266 can be simpler (aka “dumb”) and not have to be coded specifically for farmOS. The Node Red gateway device handles that (and is very good at it, too).

#7

Thanks @mstenta and @chrowe for your advice! I’ll also make the changes to the keys of my sensor on FarmOS and the https port number.

Before starting this thread, I looked into Node-RED and the resources that @chrowe has shared on these forums and was slightly confused as to how a wireless unit–like a battery-powered Arduino–could interact with such a flow.

This means that in my network I would have each sensor transmit data to the dedicated gateway device, which would then push the data through the Node-RED flow, which then uploads that data to FarmOS? Could another Arduino Uno + ESP8266 serve as that kind of gateway or would I need something with a little more processing power?

#8

Node-Red is built with node.js so you would want to run it on something like a Raspberry Pi.

I guess I never documented how to set up the http endpoint in Node-RED so it could receive data but you just use the http input node. I can try to get that set up again at some point and document better but it might not be until July.

#9

Not sure if something has been improved/fixed or if I was just not doing the right thing but https://github.com/markwbrown/farmOS-ESP8266/blob/master/esp8266sensor.ino seems to work.

#10

And slightly more useful, https://gitlab.com/fiot/HUZZAH-FarmOS/blob/master/1wire-feather_HUZZAH_ESP8266-farmos/1wire-feather_HUZZAH_ESP8266-farmos.ino takes a 1wire sensor reading and posting it to FarmOS.