Library Overview
The TWELITE Wings API (hereafter MWings) is a library for handling TWELITE from Python scripts.
Features
You can communicate with TWELITE child nodes through a TWELITE parent node connected to the host.
- Interpret received data and convert it to a dictionary, JSON, or pandas DataFrame*
- Send commands generated from dictionaries to the parent node
- The Lite version does not support pandas.
For Raspberry Pi, we recommend the Lite version instead of the standard version
- The module name is
mwingslite
, notmwings
- No dependencies on pandas, numpy, pyarrow, etc.
- On Raspberry Pi, these may not be installable from PyPI
- If pandas is not available, the function
to_df()
for outputting DataFrames will raise an exception
- As with the standard version, dictionary and JSON string output is supported
- Required Python version is lowered to 3.11 or later (standard version requires 3.12 or later)
Example Use Cases
For example, you can implement systems like the following:
- Send temperature and humidity data received by MONOSTICK to a cloud server as JSON
- Record acceleration data received by MONOSTICK to a CSV or Excel file*
- Control an LED connected to TWELITE DIP via MONOSTICK from a PC
- The Lite version cannot output directly to CSV or Excel files.
Characteristics
This is a module for modern Python, by modern Python, for modern Python.
- Installable via pip or poetry
- Supports type hints
- Validates send/receive data using pydantic
- Basically conforms to PEP8*
- With exceptions. Details explained later.
Installation
Available from PyPI.
pip の場合
pip install mwings
poetry の場合
poetry add mwings
The Simplest Sample Script
In just 6 lines, you can output received data from the Extremely Simple! Standard App (App_Twelite) in JSON format.
import mwings as mw
twelite = mw.Twelite(mw.utils.ask_user_for_port())
@twelite.on(mw.common.PacketType.APP_TWELITE)
def on_app_twelite(packet):
print(packet.to_json())
twelite.start()
Lite版の場合
import mwings as mw
with import mwingslite as mw
for the Lite version.Environment Setup and Operation Check
What You Need
- PC
- MONOSTICK (Parent and Repeater App / default settings)
- TWELITE DIP (Extremely Simple! Standard App / default settings)
- Connect peripherals such as switches (example: connect a tact switch between DI1 port and GND)
Environment Setup
The following is just one example. Any environment where Python 3.12 or later (3.11 or later for Lite version) is available is fine.
If you already have a preferred environment, feel free to skip ahead.
We take no responsibility for the use of external tools.
Also, please refrain from asking questions about external tools.
Installing pyenv
To manage the Python interpreter version, install pyenv.
Linux
curl https://pyenv.run | bash
You may need to install development tools in advance
Debian-based
sudo apt update; sudo apt install build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev curl git libncursesw5-dev xz-utils tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev
Fedora-based
yum install gcc make patch zlib-devel bzip2 bzip2-devel readline-devel sqlite sqlite-devel openssl-devel tk-devel libffi-devel xz-devel
Reference: Home · pyenv/pyenv Wiki
macOS
brew update
brew install pyenv
Install Homebrew if necessary
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
You may need to install development tools in advance
Windows
There is no pyenv for Windows. Instead, use pyenv-win.
Invoke-WebRequest -UseBasicParsing -Uri "https://raw.githubusercontent.com/pyenv-win/pyenv-win/master/pyenv-win/install-pyenv-win.ps1" -OutFile "./install-pyenv-win.ps1"; &"./install-pyenv-win.ps1"
Installing Python with pyenv / pyenv-win
Install Python 3.12 or later (3.11 or later for Lite version), as required by MWings.
To list available versions, run:
pyenv install -l
For example, to install Python 3.12.4 and apply it system-wide:
pyenv install 3.12.4
pyenv global 3.12.4
To list installed versions, run:
pyenv versions
Installing pipx
To manage command-line tools such as poetry in an isolated environment, install pipx.
Linux
Debian-based
sudo apt update
sudo apt install pipx
pipx ensurepath
From pip (recommended for Raspberry Pi)
python3 -m pip install --user pipx
python3 -m pipx ensurepath
Fedora-based
sudo dnf install pipx
pipx ensurepath
Reference: Installation - pipx
macOS
brew install pipx
pipx ensurepath
Windows
scoop install pipx
pipx ensurepath
Install Scoop if necessary
Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
Invoke-RestMethod -Uri https://get.scoop.sh | Invoke-Expression
Installing poetry
To manage the Python interpreter version and module dependencies for your project (similar to Node.js), install poetry.
pipx install poetry
If you get an error
If you get an error such as TypeError: __init__() got an unexpected keyword argument 'encoding'
, try installing pipx with pip instead of a package manager.
python3 -m pip install --user pipx
python3 -m pipx ensurepath
Creating a Project
Here, we will use mwtest
as the project name.
Move to the directory where you want to create the project and run:
poetry new mwtest
This will generate the mwtest
directory.
Project Setup
Move into the project directory and link it to the Python version you installed earlier with pyenv.
poetry env use 3.12.4
Install MWings.
poetry add mwings
If unresponsive on Linux
Try setting the following environment variable:
export PYTHON_KEYRING_BACKEND=keyring.backends.null.Keyring
Creating the Simplest Sample Script
First, let’s try running the script introduced earlier.
import mwings as mw
twelite = mw.Twelite(mw.utils.ask_user_for_port())
@twelite.on(mw.common.PacketType.APP_TWELITE)
def on_app_twelite(packet):
print(packet.to_json())
twelite.start()
Lite版の場合
Raspberry Pi のserial0
を使う例を次に示します。
import mwingslite as mw
twelite = mw.Twelite("/dev/ttyS0")
@twelite.on(mw.common.PacketType.APP_TWELITE)
def on_app_twelite(packet):
print(packet.to_json())
twelite.start()
なお pyserial の制約により、
mw.utils.ask_user_for_port()
は/dev/ttyS0
を検知できません。
上記の内容で poetry が生成した __init__.py
と同じ階層に simple.py
を作成します。
📁 mwtest
└ 📁 mwtest
├ 📄 __init__.py
└ 📄 simple.py
Running the Simplest Sample Script
Connect MONOSTICK and run:
poetry run python simple.py
If there are multiple serial ports, please select the serial port.
If you get output in JSON string format, you have succeeded. An example of actual output is shown below.
{
"time_parsed": "2024-02-20T03:16:50.150386+00:00",
"packet_type": "APP_TWELITE",
"sequence_number": 13699,
"source_serial_id": "810E0E23",
"source_logical_id": 120,
"lqi": 84,
"supply_voltage": 3249,
"destination_logical_id": 0,
"relay_count": 0,
"periodic": true,
"di_changed": [
true,
true,
false,
false
],
"di_state": [
false,
false,
false,
false
],
"ai_voltage": [
8,
272,
1032,
112
],
"mwings_implementation": "python",
"mwings_version": "1.0.0",
"hostname": "silverstone.local",
"system_type": "Darwin"
}
Contents of the JSON string
mwings.parsers.app_twelite.ParsedPacket
Key | Value |
---|---|
time_parsed | Reception time (default UTC, ISO8601 format) |
packet_type | Packet type |
sequence_number | Sequence number (time for App_Twelite) |
source_serial_id | Sender serial ID |
source_logical_id | Sender logical device ID |
lqi | Radio communication quality (8bit) |
supply_voltage | Supply voltage (mV) |
destination_logical_id | Destination logical device ID |
relay_count | Relay count |
periodic | Whether this is a periodic transmission |
di_changed | Whether each digital interface input changed |
di_state | State of each digital interface input |
ai_voltage | Input voltage of each analog interface |
mwings_implementation | MWings implementation (for future info) |
mwings_version | MWings version |
hostname | Name of the receiving host |
system_type | Type of system of the receiving host |
Creating a Practical Script
simple.py
is just an explanatory sample. It is not a practical script.
This is because twelite.start()
creates a thread to receive data, but does not provide a way to terminate it. It is also a very hard-to-read script.
Next, let’s create a more practical script. After running it, we will explain its contents.
This time, we will set the following three conditions:
- Make it possible to properly terminate the thread with
Ctrl+C
- Conform to PEP8 (with some exceptions, details explained later)
- Introduce type hints
Below is an example applying these points.
# -*- coding:utf-8 -*-
# Written for Python 3.12
# Formatted with Black
# MWings example: Receive data, print JSON, typed
from zoneinfo import ZoneInfo
import mwings as mw
# Main function
def main() -> None:
# Create a twelite object
twelite = mw.Twelite(mw.utils.ask_user_for_port())
# Use JST for received data
twelite.set_timezone(ZoneInfo("Asia/Tokyo"))
# Register an event handler
@twelite.on(mw.common.PacketType.APP_TWELITE)
def on_app_twelite(packet: mw.parsers.app_twelite.ParsedPacket) -> None:
print(packet.to_json(verbose=True, spread=False))
# Start receiving
try:
# Set as daemon thread
twelite.daemon = True
# Start the thread, Join to the main thread
twelite.start()
print("Started receiving")
while True:
twelite.join(0.5)
except KeyboardInterrupt:
# Stop the thread
print("Flushing...")
twelite.stop()
print("Completed")
if __name__ == "__main__":
# Call the main function
main()
For the Lite version
import mwings as mw
to import mwingslite as mw
.Please save the above as practical.py
.
📁 mwtest
└ 📁 mwtest
├ 📄 __init__.py
├ 📄 simple.py
└ 📄 practical.py
Running the Practical Script
Run the following command to get output in JSON format, just like with simple.py
.
poetry run python practical.py
However, this time you can exit with Ctrl+C
without causing an error, and time_parsed
should be in Japan Standard Time.
Explanation of the Practical Script
Code Explanation
Let’s explain practical.py
.
import
Statements
In practical.py
, two modules are imported:
from zoneinfo import ZoneInfo
import mwings as mw
zoneinfo.ZoneInfo
is used to specify the time zone for the received timestamp.mwings
is the MWings library. It is imported asmw
for brevity. For the Lite version, usemwingslite
.
Creating the Object
The mw.Twelite
object serves as the interface to access the TWELITE parent node connected to the host.
# Create a twelite object
twelite = mw.Twelite(mw.utils.ask_user_for_port())
The mw.utils.ask_user_for_port()
function gets a list of available serial ports on the host and returns the file descriptor path or COM port name selected by the user.
How to explicitly specify a serial port
Pass the file descriptor path or COM port name directly to the mw.Twelite
constructor.
# Linux
twelite = mw.Twelite("/dev/ttyUSBx")
# macOS
twelite = mw.Twelite("/dev/cu.usbserial-MWxxxxxx")
# Windows
twelite = mw.Twelite("COMx")
If not using a serial port
For purposes such as interpreting a log file, specify None
.
twelite = mw.Twelite(port=None)
Setting the Time Zone
By default, the reception time of data is treated as UTC.
In practical.py
, this is set to JST.
# Use JST for received data
twelite.set_timezone(ZoneInfo("Asia/Tokyo"))
Pass an IANA time zone identifier to ZoneInfo
.
Registering a Receive Handler
To process data sent from TWELITE child nodes, register a receive handler. Here, in the receive handler for the Extremely Simple! Standard App, the received data is converted to JSON format and output.
# Register an event handler
@twelite.on(mw.common.PacketType.APP_TWELITE)
def on_app_twelite(packet: mw.parsers.app_twelite.ParsedPacket) -> None:
print(packet.to_json(verbose=True, spread=False))
You can register a receive handler by applying the twelite.on()
decorator ? to any function.
mw.Twelite
object (in main()
in practical.py
). By restricting where the handler is defined, this encourages avoiding unnecessary pollution of the global namespace.Specifying packet types in receive handlers
The content of the data received by the handler is based on data classes defined according to the packet type.
Extremely Simple! Standard App
@twelite.on(mw.common.PacketType.APP_TWELITE)
def foobar(packet: mw.parsers.app_twelite.ParsedPacket):
# handle packets
Remote App
@twelite.on(mw.common.PacketType.APP_IO)
def foobar(packet: mw.parsers.app_io.ParsedPacket):
# handle packets
ARIA (Normal)
@twelite.on(mw.common.PacketType.APP_ARIA)
def foobar(packet: mw.parsers.app_aria.ParsedPacket):
# handle packets
CUE App (Normal)
@twelite.on(mw.common.PacketType.APP_CUE)
def foobar(packet: mw.parsers.app_cue.ParsedPacket):
# handle packets
CUE App (PAL Move / Dice)
@twelite.on(mw.common.PacketType.APP_CUE_PAL_EVENT)
def foobar(packet: mw.parsers.app_cue_pal_event.ParsedPacket):
# handle packets
PAL Motion / CUE App (PAL Continuous)
@twelite.on(mw.common.PacketType.APP_PAL_MOT)
def foobar(packet: mw.parsers.app_pal_mot.ParsedPacket):
# handle packets
Environment PAL
@twelite.on(mw.common.PacketType.APP_PAL_AMB)
def foobar(packet: mw.parsers.app_pal_amb.ParsedPacket):
# handle packets
Open/Close PAL / ARIA & CUE App (Open/Close PAL)
@twelite.on(mw.common.PacketType.APP_PAL_OPENCLOSE)
def foobar(packet: mw.parsers.app_pal_openclose.ParsedPacket):
# handle packets
Serial Communication (Format A, Simple)
@twelite.on(mw.common.PacketType.APP_UART_ASCII)
def foobar(packet: mw.parsers.app_uart_ascii.ParsedPacket):
# handle packets
Serial Communication (Format A, Extended)
@twelite.on(mw.common.PacketType.APP_UART_ASCII_EXTENDED)
def foobar(packet: mw.parsers.app_uart_ascii_extended.ParsedPacket):
# handle packets
ACT
@twelite.on(mw.common.PacketType.ACT)
def foobar(packet: mw.parsers.act.ParsedPacket):
# handle packets
The data class ParsedPacket
received by the handler has methods such as to_json()
to convert to a JSON string, to_dict()
to convert to a dictionary, and to_df()
to convert to a pandas DataFrame.
Here, the to_json()
method is used to convert to a JSON string.
print(packet.to_json(verbose=True, spread=False))
About optional arguments
If the verbose
option is set to False
, system information such as system_type
will not be output. If the spread
option is set to True
, List-like elements such as di_state
(of type mw.common.CrossSectional[T]
) will be expanded and output as individual fields. However, time series data such as acceleration samples (of type mw.common.TimeSeries[T]
) are not expanded.
Note that the spread
option does not exist for to_df()
. Non-time-series List-like data are always expanded into separate columns, and time-series data are expanded into separate rows.
Starting and Stopping Reception
mw.Twelite
inherits from threading.Thread
.
In practical.py
, twelite.start()
starts the receive process in a separate thread, and twelite.stop()
stops it.
# Start receiving
try:
# Set as daemon thread
twelite.daemon = True
# Start the thread, Join to the main thread
twelite.start()
print("Started receiving")
while True:
twelite.join(0.5)
except KeyboardInterrupt:
# Stop the thread
print("Flushing...")
twelite.stop()
print("Completed")
Setting twelite.daemon
to True
makes the receiving subthread a daemon. When all non-daemon threads have exited, the whole Python program will also exit. Here, the main thread is kept waiting by repeatedly calling twelite.join()
.
threading.Thread
, see the official Python documentation.When the main thread detects Ctrl+C
input, it calls twelite.stop()
in the except
block to stop the receive process. twelite.stop()
waits until the subthread finishes calling the receive handler.
Note about join()
If you do not use a while
loop as above, Ctrl+C
may not be accepted on Windows.
try:
...
twelite.join()
except KeyboardInterrupt:
...
Supplementary Explanation
PEP8 Compliance
practical.py
and the MWings source code are formatted using the Black code formatter, which is compatible with PEP8.
Strictly speaking, it is not fully PEP8 compliant
There are at least two violations:
- PEP8 specifies a maximum line length of 79 characters, but Black’s default is 88 characters. This is to avoid lowering code quality by shortening variable names just to fit lines. Reference: PyCon2015 talk
- PEP8 suggests avoiding encoding declarations, but the encoding declaration is intentionally included. In Japanese environments, there is a higher risk of non-UTF-8 files mixing in than in English environments; also, MWings is developed using Emacs, which uses encoding declarations. Black does not detect this, perhaps because it is “should not have”.
- PEP8 also contains other recommendations open to debate, such as discouraging
len()
for checking non-empty sequences (thanks to James Powell for his insights at PyCon JP 2024).
Except for the maximum line length, Black does not accept any configuration, making it a stubborn formatter, but it saves the trouble of formulating and sharing coding standards. Coding rules are often contentious, but spending resources on such trivial matters rather than on productive work goes against the Python spirit.
To add Black to your project, run:
poetry add --group dev black
dev
group indicates it is a development dependency, similar to devDependencies
in Node.js.You can run it on specific files or directories:
poetry run black mwtest
You can also check without formatting:
poetry run black --check mwtest
Type Hint Support
practical.py
and the MWings source code support type hints.
What are type hints?
Type hints were introduced in Python 3.5. Type annotations in dynamically typed Python have no effect at runtime, but static type checkers can use them to improve code quality and reliability.
Libraries that do not support type hints will cause errors with static type checkers. The previous PAL Script did not support type hints.
The MWings library uses mypy, the official static type checker for Python.
To add mypy to your project, run:
poetry add --group dev mypy
Like Black, you can specify files or directories to check:
poetry run mypy mwtest
Related Information
Practical Script Applications
We provide scripts that further develop practical.py
.
mwings_python/examples at main
log_export.py
Reads a text file containing output from the parent node, interprets it, saves the results as a pandas DataFrame, and finally outputs to a CSV or Excel file. Not supported in the Lite version.
Can be used as a command-line tool
poetry run python log_export.py -h
usage: log_export.py [-h] [-x] [-v] [-s] INPUT_FILE
Parse a log file for App_Wings
positional arguments:
INPUT_FILE text file contains logs from App_Wings
options:
-h, --help show this help message and exit
-x, --excel export an Excel file instead of CSV
-v, --verbose include system information
-s, --sort sort columns in the output
rx_export.py
Receives output from the connected parent node, interprets the results, saves them as a pandas DataFrame, and finally outputs to a CSV or Excel file. Not supported in the Lite version.
- If you select a CSV file, all results are saved in a single file.
- If you select an Excel file instead, the results are saved in separate sheets by packet type.
Can be used as a command-line tool
poetry run python rx_export.py -h
usage: rx_export.py [-h] [-x] [-v] [-s]
Log packets from App_Wings to csv or excel
options:
-h, --help show this help message and exit
-x, --excel export an Excel file instead of CSV
-v, --verbose include system information
-s, --sort sort columns in the output
rx_export.py
saves all received data to a pandas DataFrame at once, so it is not suitable for long-term logging. However, it supports Excel file output.rx_export_csv_durable.py
Receives output from the connected parent node, interprets the results, and appends them to CSV files by source serial ID. Not supported in the Lite version.
Can be used as a command-line tool
poetry run python rx_export_csv_durable.py -h
usage: rx_export_csv_durable.py [-h] [-v] [-s]
Log packets from App_Wings to csv, line by line
options:
-h, --help show this help message and exit
-v, --verbose include system information
-s, --sort sort columns in the output
rx_export_csv_durable.py
opens the CSV file and appends data each time it is received. Unlike rx_export.py
, it does not support Excel file output, but it is suitable for long-term logging.rx_print_df.py
Receives output from the connected parent node, simply converts the interpreted results to a pandas DataFrame, and outputs them as a string. Not supported in the Lite version.
rx_print_dict.py
Receives output from the connected parent node, simply converts the interpreted results to a dictionary, and outputs them. Also supported in the Lite version.
rx_print_json.py
Receives output from the connected parent node, simply converts the interpreted results to a JSON string, and outputs them. Also supported in the Lite version.
tx_binary_uart.py
Sends binary data [0xBE, 0xEF]
to TWELITE UART via the connected parent node running the Serial Communication App. Also supported in the Lite version.
tx_blink_dip_led.py
Blinks the LED connected to the DO1 port of TWELITE DIP via the connected Parent and Repeater App parent node. Also supported in the Lite version.
tx_blink_pal_notice.py
Turns on the LED of the Notification PAL in each color via the connected Parent and Repeater App parent node. Also supported in the Lite version.