Basics of Embedded and IoT Development with Python
Hi guys,
I firstly prepared this this article for a talk at Python Conference in Tanzania which took place on December 2020 at SeedSpace, Daresalaam Tanzania and originally published at GitHub with all the source presented.
I did some pre-editing to it so as it can easily be understandable by people who have did not have a chance to attend the event, I recommend you to read all the way to the end to effectively grasp the concepts I have presented;
I have background in Mechatronics Engineering where when it comes to interacting with embedded devices C/C++ becomes inevitable and at the same time I enjoy expressing my programming ideas/logic in Python rather than C/C++ .
Therefore I did some research on ways which high level language like Python can be used in embedded + IoT Development and thus what I'm going to share with you throughout this article.
Python on embedded software Development .
One of the unfair advantage low level language like C/C++ have over python is the fact they're able to interact directly with memory and also compiled down to executable binary which is small enough to be run on embedded devices with resource constraints.
While on the other side Python is an interpreted language with auto memory management using garbage collector, and also it cannot run independently of the Interpreter, therefore to run your Python code you're supposed to have an Python interpreter installed/embedded on the device you're running, Here why ?
When you write and run Python code, its firstly compiled down to intermediate byte code (.pyc) and then this byte code is does not get compiled to binary instead it It get interpreted by the Python Virtual Machine(VM) to an output.
There are several Opensource projects that focuses on porting Python to Embedded devices by reducing the size of of Python Interpreter to be shipped together with source code to those embedded devices, Here some of them;
What is PyMite ?
``PyMite is a flyweight Python interpreter written from scratch to execute on 8-bit and larger microcontrollers with resources as limited as 64 KiB of program memory (flash) and 4 KiB of RAM. PyMite supports a subset of the Python 2.5 syntax and can execute a subset of the Python 2.5 bytecodes. PyMite can also be compiled, tested and executed on a desktop computer.
Considering PyMite is designed to run on embedded 8-bit architectures, one should not expect PyMite to do everything Python can do. However, its capability may surprise you. Since PyMite is written from scratch, it has a few features that set it apart: stackless from the start and fast, easy, native C function support.`` referencing official docs
PyMite kinda old and based on python 2, and we all know the legacy of python 2 (Python 2.7) support ended in late 2020, therefore I would not recommend it unless you're willing to tinker with it then good luck.
What is MicroPython ?
``MicroPython is a lean and efficient implementation of the Python 3 programming language that includes a small subset of the Python standard library and is optimised to run on microcontrollers and in constrained environments.
The MicroPython pyboard is a compact electronic circuit board that runs MicroPython on the bare metal, giving you a low-level Python operating system that can be used to control all kinds of electronic projects.
MicroPython is packed full of advanced features such as an interactive prompt, arbitrary precision integers, closures, list comprehension, generators, exception handling and more. Yet it is compact enough to fit and run within just 256k of code space and 16k of RAM`` by micropython official docs
What is Circuit Python?
Lastly we have CircuitPython which is basically a fork on MicroPython Implementation with with few things improved Here some of them;
- Native USB Support (Circuit Python supports native USB allowing you to easily upload and update the codebase without using any special tools,you can achieve it just by copying and paste your file into the board drive as if you're transfering normal files)
- Auto-reload (Circuit Python automatically reload and run your code as save new change to its drive, saving you time of re-uploading everytime you add a new change, you just save/paste codebase it will automatically reload and run for you
- Does not support concurrency
- Error messages are translated to more than 10 languages
See at CircuitPython for more information on differences;
Going for Arduino boards
Due to availability of boards which runs python natively based on light weight implementation of Python was hard during the time of conference, I twisted a bit the presentation covering how Python can be used to control one of the popular experimental boards Arduino.
what is Arduino ?
If this is your first time hearing about Arduino boards, well its a programmable prototyping board that is heavily used by different kinds of people from professionals, teachers , students , hobbyist and ... to learn and build micro-controller based electronics projects in easy way.
It has a huge community support with tons of libraries published, and its now being used by different startup to build their MVP electronics projects therefore knowing how deal and control it might open countless of other opportunities.
Apart from the fact Arduino is used as prototyping board, its now being employed to industrial grade to control industrial process, with Development of improved board such as Controllino simplify the whole process when it comes to industrial Automation and Control using builtin Arduino board being programmed by the same arduino IDE.
How do we control Arduino with Python ?
Arduino boards cannot run python directly from the memory and space constraints we discussed above instead of controlling it directly they can be controlled with Python over USB through Serial communication in combination with some protocols, Here are some of approaches that are being done to accomplish this.
PySerial with Arduino
Installation
$ pip install pyserial
PySerial is a python library that enables the access and control of Serial port from Python, with library we can directly read the data sent from Arduino board over USB and also write to it.
In order to control a board using PySerial, we need tell python a specific port we are gonna be reading and writing to, this can be done in easy syntax just as shown below.
Using PySerial you might need to get familiar with both python and Arduino language, whereby you're going to use Arduino language to
read signals from
int led = 13;
void setup(){
pinMode(led, OUTPUT);
Serial.begin(9600);
}
void loop(){
if (Serial.available()){
char command = Serial.read();
if (command=='o'){
digitalWrite(led, HIGH);
}
else{
digitalWrite(led, LOW);
}
}
}
The above Arduino code is written in such a way it will be waiting for a signal through Serial port to control the builtin light emitting diode, the Python code to blind the LED would normally look like this.
import time
from serial import Serial
arduino_board = Serial('/dev/ttyUSB0', baudrate=9600, timeout=1)
def blink():
while True:
arduino.write(b'o')
time.sleep(1)
arduino.write(b'f')
time.sleep(1)
blink()
The above example how to write/control arduino elements from python, you can also read the signal from sensors connected to the arduino board using the similar technique, for instance let's try reading a light intensity of and LDR connected to the arduino pin A0;
The arduino-code would look like this
int ldr = A0;
void setup(){
pinMode(ldr, INPUT);
Serial.begin(115200);
}
void loop(){
int intensity = analogRead(ldr);
if (Serial.available()){
char command = Serial.read();
if (command=='r') Serial.println(intensity);
}
}
The above arduino will return the state of light intensity to an arduino when it recieve a r command other wise it will do nothing.
On the other side the Python code to read the sensor signal sent by arduino is going to look like this
from serial import Serial
arduino = Serial('/dev/ttyUSB1', baudrate=115200, timeout=1)
arduino.flush
while True:
arduino.write(b'r')
light_intensity = arduino.readline().decode()
print(light_intensity)
When to use PySerial ?
I recommend to use PySerial when something you want to control is bit complex and required other arduino libraries to run, therefore writting a code in native arduino language and providing an interface to python using Serial communication would be
better approach since the other mentioned approaches use standard firmata protocols which only provide basic control to the arduino,
If you want a detailed description to PySerial with arduino, I recommend you checking out this repo ArduinoPySerial_Series
PyFirmata with Arduino
Installation
$ pip install pyFirmata
PyFirmata is a python libary that alllow python to communicate with arduino over USB using standard firmata protocol, standard firmata, to get started you just need to upload the standard firmata code to the arduino board.
Here is link to standard-firmata-code
Once you upload the code to the arduino board you're now ready to get started controlling it using pyfirmata library just as shown below
Here is python-code to blink the LED
import time
from pyfirmata import Arduino
board = Arduino('/dev/ttyUSB0')
def blink():
while True:
board.digital[13].write(1)
time.sleep(1)
board.digital[13].write(0)
time.sleep(1)
blink()
The above code will be interpreted in real-time by the firmata protocol running on arduino board and use it to perform corresponding action, for instance for now it's going to blink the LED.
Here is a python-code to read an LDR with firmata protocols
import time
from pyfirmata import Arduino, util
board = Arduino('/dev/ttyUSB0')
util.Iterator(board).start()
board.analog[0].enable_reporting()
while True:
light_intensity = board.analog[0].read()
print(light_intensity)
time.sleep(1)
For more information on PyFirmata you can visit their official PyFirmata Documentation
Arduino-python3
Installation
$ pip install arduino-python3
Arduino-python3 A light-weight Python library that provides a serial bridge for communicating with Arduino microcontroller boards, It is written using a custom protocol, similar to Firmata.
To get started using Arduino command API in python you might need to firstly upload the prototype protocol code to the arduino and then you're on the move to begin using python to control it.
Here is a link to Arduino-commnad-api-protocol
Blinking an LED with Arduino command API would normally look like this
import time
from Arduino import Arduino
board = Arduino()
board.pinMode(13, "OUTPUT")
while True:
board.digitalWrite(13, "HIGH")
time.sleep(1)
board.digitalWrite(13, "LOW")
time.sleep(1)
The above code will automatically find the port an arduino is connected and connect to it and then begin controlling the arduino using its own custom protocol similar to that of firmata.
Here is the Arduino code to read signal from LDR using custom protocol
import time
from Arduino import Arduino
board = Arduino()
while True:
light_intensity = board.analogRead(0)
print(light_intensity)
time.sleep(0.1)
Arduino-command-API can be one with the easiest syntax of all the alternative so it all comes to you which one you choose to build your Embedded solution, for more info visit their github repository
IoT Development with Python
Now that we have seen the python space on embedded devices, its to have a look on how python also play a great role when it comes to making those devices talk to cloud and each other.
When it comes to IoT Development there main dominating protocol being used which are;
- MQTT (Message Queue Telementry Transport)
- Web Sockets
- HTTP
In a talk due to time I showed how you can do that using HTTP which can easily implemented using web framework other people are familiar with such as Django, Flask and FastAPI but when it comes into reality MQTT has now become the dominant player when it comes to implementing IoT solutions
In our today workshop we are going to use Flask to connect our devices to the internet.
Installation
$ pip install flask
If you're new to Flask, well its micro-web framework based on jinja2 templating engine ,It is very intuitive and easy to get started even If you're new you can understand the concepts about it in just a few minutes
A very simple Flask Gateway might look like this
from flask import Flask
app = Flask(__name__)
@app.route('/')
def Gateway():
return {
'who_am_i': 'I\'m a very simple IoT Gateway'
}
if __name__ == '__main__':
app.run(debug=True)
Above is our simple web-server made with flask, once you run you should see result to what shown below;
kalebu@kalebu-PC IoT -> python3 hello_world.py
* Serving Flask app "hello_world" (lazy loading)
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: on
* Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
* Restarting with inotify reloader
* Debugger is active!
* Debugger PIN: 133-374-948
Now that we have seen how a Simple flask work now led make a simple control, that allows us to control the light emitting diode from browser
Now we know how flask work let's make a simple program which gonna allow us to easily control the light emitting diode from web, in this example I have used Arduino command API but in fact you could use any of your favorite library.
with Arduino command API our Gateway code is going to look like this
from flask import Flask
from Arduino import Arduino
board = Arduino()
app = Flask(__name__)
board.pinMode(13, "OUTPUT")
@app.route('/<command>')
def Gateway(command):
if str(command) == 'on':
board.digitalWrite(13, "HIGH")
response = "The led is on"
elif str(command) == 'off':
board.digitalWrite(13, "LOW")
response = "The led is off"
else:
response = 'Uknown command'
return {'response': response}
if __name__ == '__main__':
app.run(debug=True)
Demo project
We can even something more interesting with it, the limit is your own imagination, now lets make a simple python which
we can ask it how it's outside using voice and then reply back to us.
Installation
$ pip install SpeechRecognition
In this example I have used Speech Recognition library but you don't need to know all of it just made a wrapper you can just call a listen function and then it will will do the rest.
ghost package is found in the IoT folder you can just copy and make sure its in your project directory or if you cloned the repository the folder and source code will be automatically downloaded.
Here is our code to act as minimal assistance over voice
from Arduino import Arduino
from ghost import ghost_ear
board = Arduino()
def check_outside():
for _ in range(1000):
intensity = board.analogRead(0)
intensity = board.analogRead(0)
return intensity
def Eve_Soul():
while True:
print(board.analogRead(0))
something = ghost_ear.listen()
if something:
outside_combo = ['outside', 'light', 'how', 'is outside', 'how is']
if any(word for word in something.split() if word in outside_combo):
outside = check_outside()
print('outside intensity ', outside)
if outside > 0:
print('It\'s not that dark')
board.digitalWrite(13, 'LOW')
continue
print('Outside is dark, I\'m switching light on')
board.digitalWrite(13, "HIGH")
continue
print('It\'s great serving you what can I help?')
if __name__ == "__main__":
Eve_Soul()
Here a link to the github
Hope you find it interesting , then subscribe to this blog that you never miss any upcoming articles