#255

Ғылым және технология

Grafana, InfluxDB, and Node-Red on a Raspberry Pi form a dream team for visualization of IOT data. The setup is not easy, but this is what we will do together.
Today
- We will install everything necessary on a Raspberry Pi using Peter Scargill's newest script
- We will compare time-series databases like InfluxDB with SQL databases and see their advantages for IOT
- We will create an influxDB database and connect it to a Node-Red flow to read data of my weather station
- We will create a Grafana data source to get access to the InfluxDB database
- We will develop a dashboard with several panels showing the different weather data
- We will see the power of a time-series database combined with Grafana
- And in the end, you can download a ready-made SD card for your Raspberry
Links:
SD card image with Node-Red, InfluxDB, Grafana:
drive.google.com/open?id=1MnJ...
Node-REd example Flow: drive.google.com/open?id=1u2y...
Peter's Script: tech.scargill.net/the-script
Raspbian installation: hackernoon.com/raspberry-pi-h...
Grafana videos: / @earlyadopterme9292
Samsung SD cards: s.click.aliexpress.com/e/90y5IJW
Raspberry Pi 3+: bit.ly/2tNdULR
#NodeRed, #InfluxDB, #Grafana, #RaspberryPi
Supporting Material and Blog Page: www.sensorsiot.org
Github: www.github.com/sensorsiot
My Patreon Page: / andreasspiess
Alternative: flattr.com/@andreasspiess
My Bitcoin address: 19FSmqbBzb5zsYB1d8Bq4KbxVmezToDNTV
If you want to support the channel, please use the links below to start your shopping. No additional charges for you, but I get a commission (of your purchases the next 24 hours) to buy new stuff for the channel
For Banggood bit.ly/2jAQEf4
For AliExpress: bit.ly/2B0yTLL
For ebay.com: ebay.to/2DuYXBp
profile.php?...
/ spiessa
www.instructables.com/member/...
Please do not try to Email me or invite me on LinkedIn. These communication channels are reserved for my primary job

Пікірлер: 592

  • @HendraKusumahiot
    @HendraKusumahiot5 жыл бұрын

    "We like programming, but we love result" Quote of the day. Thank you sir

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    You are welcome!

  • @bigheat86

    @bigheat86

    5 жыл бұрын

    Even I can confirm, and I'm a developer.

  • @lightpixeldotnet

    @lightpixeldotnet

    3 жыл бұрын

    this :]

  • @richard_wenner
    @richard_wenner5 жыл бұрын

    Andreas - another perfectly formed video - in every aspect. We appear to run parallel lives as I was working on this for my open air pollution monitoring project. There must be more than 24 hours in the Swiss day as I can't for the life of me see how you manage to produce a) so much high quality material, b) over such a diverse range of subjects c) in such depth in such a short space of time and d) in a Swiss accent!!!).

  • @AlenTuzlak

    @AlenTuzlak

    5 жыл бұрын

    I agree 100%. Andreas is amazing!! Thank you so much Sir for sharing knowledge with everyone and great commitment on making amazing videos !!

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    We are slow here. That is probably why our time runs a little slower... I hope the content still helps you a little for your pollution project!

  • @richard_wenner

    @richard_wenner

    5 жыл бұрын

    @@AndreasSpiess Yes, it will. The issue is the simplest way of amassing JSON over https from many disparate sources. Not sure how to organise the secure apis.

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    Security is not my specialty, fortunately ;-)

  • @opsahle2
    @opsahle25 жыл бұрын

    Extremely usefull video - I am very happy to be one of your Patrion-supporters! Keep it coming, Andreas!

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    Thank you for your support! It is always good to read that my videos are useful...

  • @aux789
    @aux7895 жыл бұрын

    Amazing! I had to stop the video on two occasions I was so overwhelmed by the sheer power of this suite of FREE software! Thanks Andreas!

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    That is the power of KZread. You can use it at your pace ;-)

  • @noealves8400
    @noealves84004 жыл бұрын

    Mr Spiess, my deepest gratitude for your work and for sharing great content in a video like this. I became your Patreon today to support your work!

  • @AndreasSpiess

    @AndreasSpiess

    4 жыл бұрын

    Thank you for supporting the channel. I appreciate it!

  • @springwoodcottage4248
    @springwoodcottage42485 жыл бұрын

    Super useful! Displaying data in a meaningful & useful way is always a great challenge between providing what is of interest to a range of user interests & storing what is needed to create this output in such a way that it is easily accessed & does not need a lot of storage. There were several great ideas within this both as shown with this software or for writing ones own software on different hardware. Thank you for sharing!

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    You are welcome. And I agree that retention strategies are not easy to define these days with very cheap disk space...

  • @mikethezipper
    @mikethezipper5 жыл бұрын

    You are seriously amazing. Thank you so much for this video - I've spend days trying to get this to work, and with just a couple hours and following your video, I've got everything up and running.

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    Glad to read that!

  • @AndreasSpiess
    @AndreasSpiess5 жыл бұрын

    The timestamps of the example data are from 2019-02-06. Please choose an appropriate date range in Grafana if you want to display them

  • @voltlog
    @voltlog5 жыл бұрын

    This set of tools is very flexible, I am using it myself and I was amassed at how easy it was to add a new device and process it's data once the whole system is configured. In my case I didn't have to go through the setup process because I just used a ready made raspberry pi image from the BigClown IoT bundle.

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    I think it is not so easy if you have to do everything from the ground. So we both were happy we were able to stand on the shoulders of others ;-)

  • @mrtarantl
    @mrtarantl5 жыл бұрын

    Amazing video for home use 😇 I have a similar setup running in my new flat, including Gas-Heater control and a combination of all sorts of different mcus with sensors and actuators based on various projects with a centralized mqtt/influx/nodered/grafana server (mostly own-software, tasmota...) Keep up the amazing work - your channel is still one of the best (mostly because you don't have annoying intros and music or semi-funny jokes ;) ) Greetings from Austria

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    Thank you! Jokes are not easy in an international environment. Glad to read that you like them. I like music, but in videos like that, I hate it myself. This is why I do not use it...

  • @PeterScargill
    @PeterScargill5 жыл бұрын

    Nice to see you're making good use of "the script" Andreass.

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    I always like to stand on the shoulders of big men ;-) Makes me more productive! Like that, everybody can profit.

  • @PeterScargill

    @PeterScargill

    5 жыл бұрын

    @@AndreasSpiess Hah. I do that all the time. My script enables software like Node-Red, written by people cleverer than me!

  • @deangreenhough3479

    @deangreenhough3479

    5 жыл бұрын

    Thank you both for your wonderful work. I have a beautiful image on my Pi and it’s all working 😁🏴󠁧󠁢󠁥󠁮󠁧󠁿

  • @hoekbrwr
    @hoekbrwr5 жыл бұрын

    This video just gave me the push to convert my smart energy meter and heating data into Grafana and Influxdb. Until now I was using Highcharts and the Dashboard of Node-RED which is so difficult to setup. Now I can add historical data. Great video, thanks Andreas!

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    I transferred historical data exporting to CSV and creating the needed input strings for influxDB with Excel.

  • @santorcuato
    @santorcuato5 жыл бұрын

    I have no words Andreas. Astonishing. Thanks.

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    You are welcome!

  • @hsmptg
    @hsmptg5 жыл бұрын

    Just to remind that "Retention Policies" in InfluxDB comes connected with the concept of "Continuous Queries". Maybe in a future video you can explain that. Great video as always. Thanks!

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    I mentioned the compression, but not in detail. I do not plan a video on that because it is very special. Maybe once part of another one. Who knows?

  • @yura979
    @yura9795 жыл бұрын

    I swear, you are a Swiss mind reader or something! I was sitting here, building my Node Red + Grafana + InfluxDB setup and your video got uploaded. O_O

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    Maybe you can save some time now ;-)

  • @chuxxsss
    @chuxxsss5 жыл бұрын

    I did find another script from another person with red-node. It is handy for solar and batteries. I hated manually install parts of Peters script the .npm title. But Peter has done a great Job with this script with Mr Sharky. I love it. Peter is a great person thank goodness we have him around. The graphics I think was made by him too.

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    With his new version, you should not need any manual install...

  • @klogaraa
    @klogaraa5 жыл бұрын

    Amzaing effort from you and Pete's , keep the videos coming Andreas 👍

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    Thank you!

  • @sausagehider
    @sausagehider5 жыл бұрын

    Such a coincidence. I was up until 1:30 am this morning looking at almost the exact same thing. Excellent content, I will give it a go today. Thanks. Martin.

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    Good luck. I hope it will work for you!

  • @athanasiosstefanopoulos5211
    @athanasiosstefanopoulos52115 жыл бұрын

    What another great video, thank you Andreas! Your video inspired me for changing my implementation on how to use historical data.

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    I hope you will be successful!

  • @athanasiosstefanopoulos5211

    @athanasiosstefanopoulos5211

    5 жыл бұрын

    @@AndreasSpiess The InfluxDB was really fast and easy! I installed influxDB in a chrooted Linux on an Android TV box where I am using it as a mqtt broker and node-red administrator. Now I am testing the first data recording from my air conditioning energy meter. Next step to install Grafana. And all these thanks to you again :)

  • @wilman_studio
    @wilman_studio5 жыл бұрын

    Awesome guide to getting Grafana up and running, thank you :)

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    You are welcome!

  • @viniciusnoyoutube
    @viniciusnoyoutube5 жыл бұрын

    Fantastic job! Just what I was needing for my project. Thank you!

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    You are welcome!

  • @rgrf
    @rgrf3 жыл бұрын

    Well done you. This, #255, is my 'eureka' tutorial. Thank you for explaining why AS WELL AS how to. I've read the rest of the Internet and really should have started here. Sensors have been my professional 'thing' for decades and I can now see a better way to process and examine the data. Thank you.

  • @AndreasSpiess

    @AndreasSpiess

    3 жыл бұрын

    Maybe you also watch my videos about IOTstack...

  • @rgrf

    @rgrf

    3 жыл бұрын

    @@AndreasSpiess I did Andreas and for the first time doing useful things with this stack. More thanks. My stuff is at rogerfrost.com. Best wishes.

  • @princebanini
    @princebanini4 жыл бұрын

    this was an eye opener for me. always a good time learning from you

  • @AndreasSpiess

    @AndreasSpiess

    4 жыл бұрын

    Glad to hear it!

  • @MrAtarifreak
    @MrAtarifreak5 жыл бұрын

    Thank you for your great video. These are the projects that I can learn from a lot....

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    You are welcome!

  • @jab5569
    @jab55693 жыл бұрын

    Nice work Andreas! Thank you for putting this together, very informative.

  • @AndreasSpiess

    @AndreasSpiess

    3 жыл бұрын

    My pleasure!

  • @christosmoutevelis4896
    @christosmoutevelis48965 жыл бұрын

    Thank you , always something to learn every week in an easy way

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    You are welcome!

  • @einars899
    @einars8995 жыл бұрын

    Interesting and useful video! It seems we are heading back to remote sensing? I like that.

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    Sensing and acting based on these data is IOT for me ;-)

  • @manuelorellanahinojosa7356
    @manuelorellanahinojosa73564 жыл бұрын

    Your channel is pure gold!! Greetings from Chile!

  • @AndreasSpiess

    @AndreasSpiess

    4 жыл бұрын

    Thank you very much!

  • @Argosh
    @Argosh5 жыл бұрын

    I've been agonizing over my heating system lately (I have been using a homebrew controller for my parents house heating unit for over 10 years now). Now I can just copy your image and dump my logging data in there while using a esp32 to control all the relais and sensors. Thank you so much!

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    I hope you will be successful in transferring the data from your source into InfluxDB.

  • @thorsten9211
    @thorsten92114 жыл бұрын

    Thank You! I love how well structured your videos are. Suits my taste as german engineer ;)

  • @AndreasSpiess

    @AndreasSpiess

    4 жыл бұрын

    Thank you! Maybe you also watch the one about Docker. It simplifies the installation of those programs

  • @thorsten9211

    @thorsten9211

    4 жыл бұрын

    @@AndreasSpiess Yes i did and everything is working nicely together. :)

  • @PileofKyle
    @PileofKyle4 жыл бұрын

    You're awesome, thanks for sharing this knowledge. I've been learning a lot from you.

  • @AndreasSpiess

    @AndreasSpiess

    4 жыл бұрын

    You are welcome!

  • @followthetrawler
    @followthetrawler3 жыл бұрын

    Very nice Andreas - I've just been playing with InfluxDB and Grafans with InfluxDB fed with SNMP data by telegraf, all in docker containers. This video has given me an idea to move that on and try to connect my weather station rather than just looking at my Internet traffic :) I'll need to try to understand my stations data stream format which should be fun. Thanks for another great video and the idea...

  • @AndreasSpiess

    @AndreasSpiess

    3 жыл бұрын

    There are a few other videos about weather station hacking and Pi and Docker (IOTstack)

  • @regenerativelife991
    @regenerativelife9915 жыл бұрын

    thanks a lot for this research and experimentation. we will definitely use this for our farm.

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    I hope your application will add some value :-)

  • @laveur
    @laveur5 жыл бұрын

    Thanks for a great video! I use Influx and Grafana professionally at work to analyze metric data from a mobile application. Thankfully I have an entire team of people to manage maintaining those systems. This kind of inspires me to see if I can hack my solar system on my roof (it uses zigbee apparently) to get data into influx and match it with real time weather so I can analyze better how my solar is performing.

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    At least you would have great support for your project. Not as we "simple people" ;-)

  • @abrax5
    @abrax55 жыл бұрын

    Really inspiring ideas, thank you! I'm thinking to move my solar panel yield visualization to an influxdb and grafana stack.

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    Good idea!

  • @christopheralexander8763
    @christopheralexander87635 жыл бұрын

    Great video, Andreas. Really enjoyed this one.

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    Thank you!

  • @alanjrobertson
    @alanjrobertson2 жыл бұрын

    Really helpful explanation of the benefits of time series databases for IoT, thank you 👍👍

  • @AndreasSpiess

    @AndreasSpiess

    2 жыл бұрын

    Glad it was helpful!

  • @ajaybnl
    @ajaybnl5 жыл бұрын

    Everything new. I had watched twice to understand. I want this setup too for my home sensors. Thank you dear.

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    You are welcome!

  • @ajaybnl

    @ajaybnl

    5 жыл бұрын

    Im now successfully using grafana influxdb with mosquitto (EspMqtt) using node red on my OrangePi Lite.

  • @drfritz142
    @drfritz1425 жыл бұрын

    Thank you. This is what I needed. I'll get to work right away...

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    Enjoy!

  • @portfedh
    @portfedh3 жыл бұрын

    Thank you so mucho for this. I succesfully followed this tutorial and now have an energy monitor and a water pressure sensor working and displaying data in grafana.

  • @AndreasSpiess

    @AndreasSpiess

    3 жыл бұрын

    Excellent!

  • @ursmii_
    @ursmii_4 жыл бұрын

    I took the newest RasPi-4 & OS Buster, used your selections @2:53 and the latest version of peter's script. It took 17 minutes for a full installation. Thanks for this video

  • @AndreasSpiess

    @AndreasSpiess

    4 жыл бұрын

    Very fast! Did you try the docker Installation?

  • @ursmii_

    @ursmii_

    4 жыл бұрын

    no I did not. I'll try to create a surveillance for 24/7 running PCs (if message does not arrive periodically - alert)@@AndreasSpiess

  • @AndreasSpiess

    @AndreasSpiess

    4 жыл бұрын

    Interesting project! We created a whole project with lots of useful containers (IOTstack)...

  • @Techonsapevole
    @Techonsapevole5 жыл бұрын

    Thanks Andreas, did you consider to wrap this config in Docker/Docker Compose instead SD ? It would be easier to upgrade

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    No, not for the moment. Maybe later...

  • @MitchellWongHo
    @MitchellWongHo5 жыл бұрын

    Excellent demonstration of how little you need to collect and visualise data. I have a spare PINE64 which I'd like to try using these building blocks

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    You have to check which operating systems the script supports if you want to use it.

  • @kokoscom
    @kokoscom5 жыл бұрын

    Excellent video Andreas. Clear instructions and to the point. We have used mqtt (activeMQ) +nodered+mysql and we were planning to use influx dB since its performance is more suited to IoT and time series data. Node red is really amazing and can help you to get to the results without the need of programming. Grafana (even if its free) its excellent visualisation tool. We have everything working under Ubuntu VPS, docker images of MQTT, node red, mysql, grafana etc and the performance is more than enough (more than 25k measurements in one minute transfered from the remote sensor via mqtt to the database without any issues)

  • @kokoscom

    @kokoscom

    5 жыл бұрын

    it would be great if any of you wants to collaborate for any EU funded research based projects. (as mentioned most of us live parallel lives.. and do similar things)

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    25k points per minute are quite a lot! I did not know this is possible. Docker is a good concept. But with Peter's script, it is also quite easy to install. And we have full access to the RPI hardware...

  • @rainbowforest09
    @rainbowforest095 жыл бұрын

    This is what i'm looking for. Thanks!!

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    :-)

  • @qamarkilani551
    @qamarkilani5515 жыл бұрын

    Many Thanks Andreas - Well Done and hope to see more

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    You are welcome!

  • @ThWischi
    @ThWischi2 жыл бұрын

    Dear Andreas, great Video! Quite complex things explain so amazing short. Stuff like this can only see very rarely. Love your comments why you choose this tools. Thank You! Greets Thomas

  • @AndreasSpiess

    @AndreasSpiess

    2 жыл бұрын

    Thank you for your kind words!

  • @PhG1961
    @PhG19615 жыл бұрын

    Waw, excellent work ! Heavy stuff aswell !

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    The simple stuff is done by others ;-)

  • @Alby_VA
    @Alby_VA3 жыл бұрын

    Big thanks. I was using the Node Red Dashboard and it was awful, nevermind browser issues if you left it on the dashboard page too long. Pushing my data to InfluxDB and then viewing beautiful charts in Grafana from that DB information is awesome. Thanks,

  • @AndreasSpiess

    @AndreasSpiess

    3 жыл бұрын

    I also like Grafana...

  • @Alby_VA

    @Alby_VA

    3 жыл бұрын

    @@AndreasSpiess I would also add that I'm using Python on a (Pi Zero W) to push MQTT data to a remote server listening for MQTT and is hosting InfluxDB, Node Red, and Grafana. You just need to install Paho MQTT on the Pi (pypi.org/project/paho-mqtt/). Then in the script just import it (import paho.mqtt.client as mqtt), define the remote server (mqttBroker = 'foo.server.com'), set the client with client = mqtt.Client(), connect to the remote server, client.connect(mqttBroker), and then for every variable assign to your data feed, you then just publish it with something like this: client.publish('Temperature', temperature). Where temperature = bmp.temperature, a feed coming from a BMP390 sensor. I any case, following your instructions on Influx DB, Node Red (MQTT->InfluxDB), and Grafana has been a life saver. It sure beats trying to chart something from file data. :-)

  • @michaelgruner941
    @michaelgruner9415 жыл бұрын

    Grüezi Andreas, as always a great video with many explanation that everybody can follow and learn a lot. todays topic is very sexy, since data/process mining is the new lubricant of the world. as already told on TTN con I love your 'Mutterwitz'. cheers Michael (guy form Sulzer)

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    It was long overdue for me as I wanted to use it for my installation. I hope you had a good trip home!

  • @leedevries5660
    @leedevries56605 жыл бұрын

    Great work again Andreas!

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    Thank you!

  • @desy666
    @desy6665 жыл бұрын

    Useful and interesting as usual 😉

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    :-)

  • @YouGenom
    @YouGenom5 жыл бұрын

    Great, finally this seems to solve my IT problems for IoT. HomeAssistant is also good but seems to have an easy start with challenging future. (developing all the plugins etc. for simple additional custom features!)

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    So far I always used Node-Red and do not know Home Assistant.

  • @YouGenom

    @YouGenom

    5 жыл бұрын

    @@AndreasSpiess For whom, who might be interested... Home Assistant (HA) has MQTT capability. An MQTT broker can direct messages to HA and HA will store them and display in a nice manner. Only by writing some making configuration files, one can trigger alerts, actions etc... e.g. I could set up a threshold for humidity sensor, after crossing this level, HA triggered a relay, which controls the fan. It is neat but extending it seems to be difficult.

  • @DavidL3430
    @DavidL34305 жыл бұрын

    Great video and very timely for me as I am looking to complement my old RRD based graphics site with Grafana and Infuxdb. I have used Pete’s ( and Mr Shark’s) script along with his node-red big timer for years! While I do enjoy node red, I am looking to lower my dependence on it. Thus hints of ways to direct mqtt data directly to influxdb are high on my curiosity list!!

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    A other viewer commented he is using python scripts instead of Node-red

  • @RalfVogler

    @RalfVogler

    5 жыл бұрын

    github.com/influxdata/telegraf/tree/master/plugins/inputs/mqtt_consumer

  • @tarmiziizzuddin337
    @tarmiziizzuddin3375 жыл бұрын

    Thank you very much sir! Always love your video

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    You are welcome!

  • @lucapesce3335
    @lucapesce33355 жыл бұрын

    Cool, I use InfluxDB for my home, but never understood the retention policy, thank you. I have three point about your setup: - InfluxDB is part of the TICK stack, that has Chronograf as data visualization tool in place of Grafana, a different flavour of the same concept someone might prefer, and has Telegraf agent you can use to collect raspberry's own metrics to track its health, feeding them through the same InfluxDB and Grafana combination - This raspberry use case tends to put some pressure on the SD because of the big quantity of data written and red, it is important to consider an external HDD and a backup strategy (obviously you already know this) - All those "installations" and "SD images" are things of the past, you need to switch to the container approach, I am talking about Docker; that will give you ease of installation, configuration, resource balancing, sharing, testing different setup, replication of setup and tenths of other things you will discover

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    1. I made a video about SD cards and showed how you can enable an HDD. So far I had no problems with my SQLite database. The number of points written is not a lot. 2. As soon as Docker gives full HW access it will be a good solution also for the RPI. I had problems to access a simple USB port on my PC. Otherwise a good concept with a lot of potentials.

  • @SmithyScotland
    @SmithyScotland5 жыл бұрын

    Thank you. I've been struggling for years with mysql and an inflexible php graphing library.

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    This is maybe a little more flexible...

  • @alexanderkirchnerat
    @alexanderkirchnerat4 жыл бұрын

    Sehr geil, wie immer! Vielen Dank!

  • @AndreasSpiess

    @AndreasSpiess

    4 жыл бұрын

    Bitte, gern geschehen!

  • @Aemilindore
    @Aemilindore5 жыл бұрын

    Thanks so much for providing the SD image. Thanks thanks thanks

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    You are welcome!

  • @icecaper
    @icecaper5 жыл бұрын

    Great video again, Andreas and superb timing as I too have a project which requires this type of sensor recording/display. Have you used an IOT2020?

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    No, I usually do not use such "professional" components like the IOT2020

  • @LucasGuillemette
    @LucasGuillemette5 жыл бұрын

    I'm a programming student and I was this kind of data retention policy saves a lot of work. Glad I don't have to reinvent the wheel.

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    We live in a good time with all this open source software...

  • @superconfort
    @superconfort5 жыл бұрын

    Thanks, this was just what I needed

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    You are welcome!

  • @veljac99
    @veljac995 жыл бұрын

    Nice video Andreas! In my case, instead of manual instalation, since I started using Home Assistant a few days ago, I just added InfluxDb and Grafana pluggins in Home Assistant which was like 5 minutes effort (Node Red I already added before). Anyway your instructions helped a lot for a fast start. And a sugestion - if you didn't already - check out Home Assistant. Nice platform + combined with Mosquitto, Node Red and now InfluxDb and Grafana ... make it very interesting. I use some manually programmed microcontrolers (bathroom humidity and fan control), Nextion displays from your video over MQTT, and dwo days ago I reprogram own application for LED strip and Sonoff Basic with esphomeyaml. Addionally using Alexa over Node Red, that fits very niceli into Home Assistant. New graphs from bathroom humidity, temperature and fan control is today nice addon. Point is that any additional knowledge nicely fits in picture. Constant evolution...

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    Other commenters also mentioned that it is easy with Home assistant. Maybe I have to try it one...

  • @avejst
    @avejst5 жыл бұрын

    Fantastic project👍😀 Thanks for sharing😀👍

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    You are welcome!

  • @partscience
    @partscience5 жыл бұрын

    Wao!!!! Great idea!!!

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    :-)

  • @wei48221
    @wei482215 жыл бұрын

    Awesome as always.. Thank you...

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    :-)

  • @RicardoRodriguesPT
    @RicardoRodriguesPT5 жыл бұрын

    Another excellent video. Thanks!

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    You are welcome!

  • @alainvihumbira8354
    @alainvihumbira83544 жыл бұрын

    Thank you for the video, otherwise I have a question is it possible to integrate Grafana in a php page or is it an open source that can be easily integrated into a web page in your site?

  • @AbcDef-hl2ic
    @AbcDef-hl2ic5 жыл бұрын

    Congratulations and thank you again!

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    Thank you!

  • @MirkWoot
    @MirkWoot5 жыл бұрын

    Lovely!. I've used InfluxDB + Grafana for almost a year, but for me the confusing part is Node-Red. I've instead used a InfluxDB libary for my ESP8266 codes, there is also for Python, etc.. and a small program "Telegraf" can send data from windows, macs, and linux machines.

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    I did not know a library exists. Thanks for the tip. In my case, it is more flexible to use Node-Red as an "intermediate". And the rest (telegraph, python) is maybe for other channels ;-)

  • @steveb7600

    @steveb7600

    5 жыл бұрын

    You can easily send data our from Arduino and the ESP8266 as HTTP, ingest it in a Node-Red inout, then send it out from Node-Red in MQTT on port 1883 or 8883 depending if you need certs for TLS1.2. It can do a lot as an intermediary, as Andreas said.

  • @kwazar6725
    @kwazar67255 жыл бұрын

    Cool. Great stuff Andreas.

  • @kwazar6725

    @kwazar6725

    5 жыл бұрын

    Its all the craze doing opensauce. I wonder what made u not choose non elk and or prometheus. I suppose grafana is top for these requirements. Miss lora and esp32 a bit... a lot to be said about long range iot.

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    Thank you!

  • @user-lj9uu9zb4w
    @user-lj9uu9zb4w5 жыл бұрын

    Thank you for the hundreds of really useful, interesting, up to date tutorials and info, you're pure gold! I have a question regarding the pi node-red server side, i need to have it in a distant location and feed it with 12V batteries (car battery) and let it run node-red influx db and grafana. What would be a good way to convert 12V down to 5V for the pi? Would one of these cheap buck converters do? What do you suggest? thank you in advance!

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    A buck converter with at least 3A should do it. There are many projects around if you search with Google

  • @peopledrivemecrazy
    @peopledrivemecrazy5 жыл бұрын

    Amazing work!

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    Thank you!

  • @MehrdadGivehchi
    @MehrdadGivehchi5 жыл бұрын

    What a great video. Many thanks!!!!

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    You are welcome!

  • @gavinsmalley1513
    @gavinsmalley15135 жыл бұрын

    Another great video. By coincidence I was looking at just these products yesterday. Thank you for such a detailed but concise summary. I am left with one question though, since InfluxDB automatically timestamps data is there any way of moving already existing historical data into an InfluxDB without it all being given the current time?

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    AFAIK you can insert also timestamp fields. At least this is what I did when I copied data from one RPi to another.

  • @px1690
    @px16905 жыл бұрын

    Hi Andreas, you might want to consider hass.io which has all as point and click install packages for all you talked about today! Great also to create automation for your tv lighting etc.. all over a web gui so easy it will withstand the wife test.

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    Maybe I have to look at hass.io once...

  • @victorchorques4893
    @victorchorques48935 жыл бұрын

    Great job!! Very useful video.

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    Thank you!

  • @UpLateGeek
    @UpLateGeek5 жыл бұрын

    I started out using node-red and MongoDB, because you used to be able to get free cloud-based MongoDB services, but that time-based data consolidation is really nice, so I'll probably switch to InfluxDB for my projects from now on.

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    MongoDB seems to use quite a lot of disk space because it is made for other purposes. There is a comparison on the influxDB page...

  • @UpLateGeek

    @UpLateGeek

    5 жыл бұрын

    @@AndreasSpiess yeah, MongoDB seems to store a lot of metadata for every record, which could be useful for some things, but not so much for simple IoT projects.

  • @Ne3M1
    @Ne3M14 жыл бұрын

    Awesome video!

  • @AndreasSpiess

    @AndreasSpiess

    4 жыл бұрын

    Glad you enjoyed it

  • @asiw
    @asiw5 жыл бұрын

    This is a new concept for me (and it's very interesting), I have used a more traditional sql approach based on the work by Csongor Varga "Working with time series data in Node Red with SQLite" on KZread. Your viewers might find the comparison interesting.

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    SQL is always a possibility. It is stable and does its job.

  • @luisd101
    @luisd1015 жыл бұрын

    Very nice! The problem with the Pi for DBs is the SD. Even good SDs tend to worn out fast and you lost your valuable DB. I would recomend to store db in other disk even at the expense of use USB disk. There are other SBC with SATA/PCIe that will be more suited this purpose.

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    I once did a video about that. For me so far I had no bad experience after more than a year (not Grafana,SQLite)

  • @nikolaimorozov6383
    @nikolaimorozov63834 жыл бұрын

    Many thanks for your video!

  • @AndreasSpiess

    @AndreasSpiess

    4 жыл бұрын

    You are welcome!

  • @uweri_3851
    @uweri_38515 жыл бұрын

    Thanks for IMG file. Awesome stuff.

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    :-)

  • @TechnoAutomation
    @TechnoAutomation5 жыл бұрын

    Great explained sir.

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    Thank you!

  • @asiw
    @asiw5 жыл бұрын

    Happy birthday Andreas, hope you have a good one.

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    Yes, I had a good one. My new bike for the Danube tour arrived right that day :-)

  • @rbid
    @rbid3 жыл бұрын

    Excellent video. Thanks. Greetings from the galilee

  • @AndreasSpiess

    @AndreasSpiess

    3 жыл бұрын

    Glad you liked it!

  • @Mariete
    @Mariete5 жыл бұрын

    I would like to see also a video about continuous queries and retention policies in detail with real examples related to weather (and possibly energy data). I have about three years of data every few seconds for my many weather and energy sensors but find very cumbersome to implement retention policies and continuous queries and to downsample the already existing data.

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    I do not see that this is a topic with lots of interested people. I think you will find some information on the net. Your situation is perfect as you can play with your data and find out which policy fits your needs

  • @joakimbergstedt4390
    @joakimbergstedt43904 жыл бұрын

    Hi! Could you please upload the NR-flow? I am really interested in the details in the function-nodes. I am struggling with my own input from NR to InfluxDB. :)

  • @familyplans3788
    @familyplans37884 жыл бұрын

    just putting up xmas lights but as soon as that is done i will be following this video and cant wait to use Mr. Scargills 'script'

  • @AndreasSpiess

    @AndreasSpiess

    4 жыл бұрын

    Maybe you watch the „ultimate pi server“ video first.

  • @familyplans3788

    @familyplans3788

    4 жыл бұрын

    @@AndreasSpiess LOL i was just wondering that !! good job the mrs LOVES your accent so i get away with watching geeky videos

  • @eriklaken1025
    @eriklaken10255 жыл бұрын

    Nice for my next step, 3 monts from now, i donate when i download the sd card, Have a Nice Day and Thank,S for so much Help!

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    You are welcome!

  • @eriklaken1025

    @eriklaken1025

    5 жыл бұрын

    Thank You, now i have made a sensor for my garden, Very litle solarpanel, a Mini_Pro that sleeps all Day with a Ds3231 clock, wakeup Once a Day send by WiFi with a ESP8266 with turn On with mosfet and give data to mqtt. The System works and have with 18650 batery power enoff. The py and MQTT make a Nice http:). So you can do a lot with a ESP8862 in this way. No power problem. I see On my computer How dry the garden is and maby give iT Some action.

  • @karenlalayan3700
    @karenlalayan37005 жыл бұрын

    Number One!!! Thanks.

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    ??

  • @diegogonzalezz
    @diegogonzalezz4 жыл бұрын

    Great video thank to this i have a working weather station for a month now, i have some questions about the RETENTION POLICY, is the one you explain the default? do you know any source where i can find more about this? Great work on youtube I´m a long time fan.

  • @AndreasSpiess

    @AndreasSpiess

    4 жыл бұрын

    I never used retention policies so far. My Pi can handle the data volume up till now.

  • @banzobeans
    @banzobeans5 жыл бұрын

    Good stuff!

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    Thank you!

  • @90daner
    @90daner3 жыл бұрын

    you are my new god sir! thank you so much!

  • @AndreasSpiess

    @AndreasSpiess

    3 жыл бұрын

    You are welcome!

  • @daffdaffelito
    @daffdaffelito5 жыл бұрын

    A verry good video! Thanks. I all ready got influxdb and node red on my server. But i never used the influxdb node in node red befor. It would be great if your node red flows where published somewhere!!

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    They are: In the comments, you should find a link.

  • @daffdaffelito

    @daffdaffelito

    5 жыл бұрын

    @@AndreasSpiess thanks! Looked a bit better now and found them.

  • @stefanmarinescu5086
    @stefanmarinescu50865 жыл бұрын

    That is too complex for my level but is very nice to know that if needed, this is a good reference for learning.

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    It should be easy if you install the image of the SD card to start playing and learning...

  • @squalazzo
    @squalazzo5 жыл бұрын

    @Andreas issues are now solved, as you know by latest comments on Pete's blog... i'm MrShark :)

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    Thanks! The video was already out then. I pin your comment that everyone can read it.

  • @squalazzo

    @squalazzo

    5 жыл бұрын

    @@AndreasSpiess problems were caused by a double install of sqlite node, 1 in Pete's node selection, 1 after that with unsafe-perms, and the sqldbs node which is unmainained from 2 years, we removed it... and now nodes are installed 1 by 1, instead on a single batch, so if 1 fails, it's just that, it does not make the others fail all together... now seems all fine

  • @squalazzo

    @squalazzo

    5 жыл бұрын

    @@AndreasSpiess minor issues solved this morning: switched to official grafana repository instead of the previous one, as now it exists and it's working properly (it was not so when i added grafana and influxdb to script, a year ago or so...)

  • @MrJanulis

    @MrJanulis

    5 жыл бұрын

    @@squalazzo Influxdb node still requires manual installation - tried just a couple of hours ago

  • @DaveBrown1
    @DaveBrown12 жыл бұрын

    Excellent, very good.

  • @AndreasSpiess

    @AndreasSpiess

    2 жыл бұрын

    Thank you!

  • @PeterHellmich
    @PeterHellmich5 жыл бұрын

    Hello Andreas! Thanks for the great video. I have already tried Pete's Script some time ago and am using node red for over 2 years knnow! Worked great, so i'm doing a fresh install now on the latest Raspberry and I am adding Influx DB now for the first time. Iw wanna try to store and read power consumption data that I am transmittting from an ESP8266 in my fuse box. Have you tried anything like that yet?

  • @AndreasSpiess

    @AndreasSpiess

    5 жыл бұрын

    I made a video and an image is there for download. Today I worked on the alarming from Grafana via Telegram to the mobile.

  • @realsushi_official1116
    @realsushi_official11163 жыл бұрын

    Very interesting and easy to jump in. Actually I was using a Python Jupiter Notebook for Receiving, parsing, sending payload to the MQTT broker. This solution is great, especially for future maintenance

  • @AndreasSpiess

    @AndreasSpiess

    3 жыл бұрын

    I never thought about using Jupiter Notebook for something like that. Now I have to look into it. Tanks for the info.

Келесі