We have a dataset that challenges Spanish norms, and we are trying to fine-tune our base GPT2 model so that we can start «contaminating» the outcome, aiming at inventing new words out of this «contamination.»

Our dataset’s clean version is «Las tres mitades de Ino Moxo y otros brujos de la Amazonía,» a book by César Calvo, a Peruvian poet who wrote this book highlighting many words in the Amazonian dialect, creating poetic, fantastical, yet «incorrect» narrations.

We should experiment with various approaches, but we will begin by downloading a small GPT-2 Spanish model and using it to train on a dataset. To do this, we need to download this base model onto our environment:

python - << 'EOF'
from transformers import AutoTokenizer, AutoModelForCausalLM
model_name = "DeepESP/gpt2-spanish"
print("Loading model:", model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name, use_fast=True)
model = AutoModelForCausalLM.from_pretrained(model_name)
print("Loaded OK.")
EOF

The «EOF» syntax enables Python scripts to be written directly within the terminal window, eliminating the need to create a separate Python file. In theory, we could have written a Python file, for example, ‘download_base_model.py’ with the following inside:

from transformers import AutoTokenizer, AutoModelForCausalLM
model_name = "DeepESP/gpt2-spanish"
print("Loading model:", model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name, use_fast=True)
model = AutoModelForCausalLM.from_pretrained(model_name)
print("Loaded OK.")

And then run it by typing in our Terminal window ‘python3 download_base_model.py’. It’s equal, the same. Now that we haveloaded our base model, we can test it:

python - << 'EOF'
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
model_name = "DeepESP/gpt2-spanish"
print("Loading tokenizer/model...")
tokenizer = AutoTokenizer.from_pretrained(model_name, use_fast=True)
model = AutoModelForCausalLM.from_pretrained(model_name)
# Move to MPS if available
device = torch.device("mps" if torch.backends.mps.is_available() else "cpu")
model.to(device)
print("Using device:", device)
prompt = "Un robot desobediente es"
inputs = tokenizer(prompt, return_tensors="pt").to(device)
with torch.no_grad():
 outputs = model.generate(
 **inputs,
 max_new_tokens=60,
 temperature=0.9,
 top_p=0.95,
 do_sample=True,
 pad_token_id=tokenizer.eos_token_id,
 )
text = tokenizer.decode(outputs[0], skip_special_tokens=True)
print("\n--- OUTPUT ---")
print(text)
EOF

This asks the model to generate a maximum of 60 new tokens after the phrase (prompt) «Un robot desobediente es». The output is interesting and confirms our GPT-2 model works:

Un robot desobediente es el que se ha apoderado de la máquina, y la máquina, a fin de que no se haga daño o que la máquina deje de salir. «

«La máquina es el único superviviente de las máquinas humanas que ha sido destruida», reflexionó.

From here, we will want to begin «corrupting» our model. This task has been completed.

We have already installed Miniconda3, which is provided as the default for our Linguistic Disobedience project. If Miniconda/conda isn’t already installed on the computer to be used, then this step needs to be pursued. Knowing that we have Miniconda3, we create a new environment:

conda create -n disobedient-nlp python=3.11

We are requesting that the environment run on Python 3.11, despite Python 3.12 being the default version installed through Homebrew within the system. This is due to Python 3.11 being much safer and compatible for machine learning work (PyTorch, Transformers, PEFT, etc.).

After this step, our environment will be created. An environment is important because, within that environment, we can select options such as a different Python version, among other things, without affecting the entire system. Once the environment is created, we can verify it is installed:

conda env list

Then, we can activate it:

conda activate disobedient-nlp

In my case, once the environment is activated, it will show between ():

(disobedient-nlp) NKA-MacBook-Pro:~ nka$

Then, if we want to deactivate this environment:

conda deactivate

To configure the environment for what we want, we need to install some additional tools:

pip install torch torchvision torchaudio   # For an M1 MBP, this will automatically install using MPS (Metal Performance Shaders)

pip install transformers datasets peft accelerate sentencepiece.  # Hugging Face Tools

This task is now complete.

We want two connections to be an option, like we have with robots DR1, DR2, and DR3. For our setup using the RPi Zero 2 WH (DRX), we want to connect to the local ‘disobedientrobots’ network using our internal Wi-Fi card and, when connecting an external Wi-Fi adapter, we want to be able to access the internet. To do so, we need to modify /etc/netplan/50-cloud-init.yaml.

sudo nano /etc/netplan/50-cloud-init.yaml

Then, we need to modify to have the following:

network:
    version: 2
    wifis:
        wlx90de8023b2b9:   #the name of the external Wi-Fi adapter
            dhcp4: true
            access-points:
                "MCD-PUBLIC": {}
            optional: true
        wlan0:   # the internal Wi-Fi adapter
            dhcp4: false
            dhcp6: false
            addresses:
                - 192.168.2.111/24   # static IP address
            nameservers:
                addresses: [192.168.2.1, 8.8.8.8]
            access-points:
                "disobedientrobots":
                    password: "disobedientrobots"
            optional: true
    ethernets:
        eth0:
            optional: true
            dhcp4: true

This should work. If for some reason internet connection isn’t working, then disconnect, reconnect the external Wi-Fi adapter, and wait briefly before trying again. Our DRX-server machine is now «111».

Marked as complete.

Ubuntu Server has no GUI, meaning that it only works within the «terminal». To enable multiple «windows,» we can install tmux:

sudo apt install tmux

Then, to run tmux, we simply type in:

tmux

Ctrl+B, then C will create a new window.

Ctrl+B, then » will split the screen horizontally, while Ctrl+B, then % will split the screen vertically. Once the screen is split, we can navigate between screens using Ctrl+B, followed by the arrow keys.

This task has been completed.

After successfully installing Ubuntu 24 LTS Server using the Raspberry Pi Imager into the Pi Zero’s SD Card, we’ll need to install the «bare bones» version of ROS2 Jazzy into the system.

Before proceeding, we need to make sure we’re connected to the internet with the Pi Zero by adding the following to the /etc/netplan/50-cloud-init.yaml network config file:

network:
  version: 2
  wifis:
    wlan0:
      dhcp4: true
      access-points:
        "MCD-PUBLIC": {}   # In our case, the network name at the Newlab is "MCD-PUBLIC" and has no password, hence the {}
      optional: true

Then, following the Official Ubuntu Install Instructions for ROS2 bare bones (ros-jazzy-ros-base) should work well. It worked for us!

We had to install ros-jazzy-demo-nodes-cpp and ros-jazzy-demo-nodes-py after the fact to run the classic talker/listener test. It worked!

This task has been completed.

Following the ROS2 tutorials, we set-up the ROS_DOMAIN_ID = 1, so that it matches DR1, we are assuming we will use DRX at .111 to communicate with DR1, if this changes, we will have to change the Domain ID too.

I finally have them all compatible so that means that I can run the ros2osc bridge node in order to start playing with receiving messages within Supercollider.

Yesterday I added an additional stream of sound to the DRM, which is a slowly fading it fading out white noise, only that it is a unique source of white noise for each of the 16 loudspeakers instead of the 16 having the same source of white noise. This produces an interesting «3d» effect inside the DRM.

An interesting thing we noticed yesterday while taking pictures of the project was that inside the DRM the sounds that are coming from outside like voices or even sounds coming from AGNS robots enter the dome and transform themselves! It is an unexpected effect and at the same time very fitting with its own name as a machine that mutates the outside reality. A gift from the gods perhaps.

I will continue working on linking robot movement and sound within the DRM and will report later!

I finally have them all compatible so that means that I can run the ros2osc bridge node in order to start playing with receiving messages within Supercollider.

Yesterday I added an additional stream of sound to the DRM, which is a slowly fading it fading out white noise, only that it is a unique source of white noise for each of the 16 loudspeakers instead of the 16 having the same source of white noise. This produces an interesting «3d» effect inside the DRM.

An interesting thing we noticed yesterday while taking pictures of the project was that inside the DRM the sounds that are coming from outside like voices or even sounds coming from AGNS robots enter the dome and transform themselves! It is an unexpected effect and at the same time very fitting with its own name as a machine that mutates the outside reality. A gift from the gods perhaps.

I will continue working on linking robot movement and sound within the DRM and will report later!

This is my take; an exhibition space and time is, for me, an excellent time to continue experimenting instead of leaving work that is «finished». As a matter of fact, I think that one dimension of «disobedience» is all about considering work always as «unfinished» in the sense that it is constantly evolving. And exhibiting this way, as a way of experimenting and evolving through time, is for me an idea and perhaps a statement I will follow.

I will be writing posts as the day(s) go by, in a way this will be my exhibition journal! So I will start with what’s happening right now at the beginning:

– DRM is now supported by its own legs, they can flex and adjust to the height of different people seating below it. Adjustment isn’t automatic, rather I need to operate it so that it goes to the appropriate height. I’m not sure if automatic adjustment will happen in the future, both because it is a difficult technological feat, and because me being present as a performatic operator is also an interesting aspect to explore.

– Sound coming out of the DRM is currently at its most basic, and I hope this will evolve over the next few days. There is currently no connection between robots movement but there is a way to do it since DRM is already running in its own RPi5 with ROS2 installed. It’s just a matter of running the OSC bridge while receiving signals from the other robots.

– Robots are currently expressing the clicking sound that they had last time they were active in Seattle. This is the last bit of their memory but this will start changing as I include different recordings from CDMX and their own construction as ways to explore their expression.

– Robots movement is currently limited because coding is made for bigger spaces, they get stuck too easily. I have some ideas, for example, having robots move not all 3 at the same time (I mean navigate, they can still move up and down while parked).

I will probably write in the comments as I come up with ideas and changes! Stay tuned!!

And so it begins…

From Ethan:

One thing to note, platformio configuration will need to be done for the specific type of microcontroller you have and the version of microros/ros2 you are targeting. In this case we chose nodemcu-32s (it was the one we had) and iron for our distro, though you may be using different systems.

Then,

To do this, we will use tmux. First, install tmux:

sudo apt update
sudo apt install tmux

Then, we should enable the use of the mouse in tmux:

nano ~/.tmux.conf   # if the file doesn't exist it will be created

Inside this file, we need to add the following line:

set -g mouse on

Then close any tmux sessions, the new ones will have the mouse enabled (to activate panes, for example).

To make things even easier, we will install tmuxinator. In a Terminal window, run:

gem install tmuxinator

Then, tmuxinator needs an editor to be defined, so if ‘echo $EDITOR’ provides no answer, we can:

export EDITOR=nano

Or if we want to save this for all bash/Terminal sessions, we can:

echo 'export EDITOR=nano' >> ~/.bashrc
source ~/.bashrc

Then, we can create our first project:

tmuxinator new DR1_session

It will launch the tmuxinator config file, in which we will add:

# /home/disrobot/.config/tmuxinator/DR1_session.yml

name: DR1_session
root: ~/slam_ws
windows:
  - editor
      layout: tiled # Ensures an even tiled layout for the panes
      panes:
        - source install/local_setup.bash # roboclaw_node pending: ros2 launch roboclaw_node_ros roboclaw_node.launch.py
        - source install/local_setup.bash && ros2 launch twist_mux twist_mux_launch.py config_topics:=/home/disrobot/slam_ws/src/twist_mux/config/twist_mux_topics.yaml
        - source install/local_setup.bash && ros2 run teleop_twist_keyboard teleop_twist_keyboard --ros-args -r /cmd_vel:=/key_vel
        - source install/local_setup.bash && ros2 launch rplidar_ros rplidar_a1_launch.py
        - source install/local_setup.bash && ros2 launch rplidar_ros rplidar_a1b_launch.py serial_port:=/dev/bottom_lidar frame_id:=bottom_laser remap_scan:=/bottom_scan
        - source install/local_setup.bash && ros2 launch robot_wander startwandering.launch.py
        - source install/local_setup.bash && ros2 run ros2osc ros2osc_node
        - source install/local_setup.bash && ros2 launch lift_routine liftroutine.launch.py
        - source install/local_setup.bash && ros2 launch timer_conductor timer.launch.py
        - QT_QPA_PLATFORM=offscreen sclang ~/SC/DR1_SCvol.scd

This file is located in /home/disrobot/.config/tmuxinator/DR1_session.yml

To launch this session we need to

tmuxinator start DR1_session

And to end it

tmuxinator stop DR1_session

As we are going to be using one of the robots (101) as the master repository from which 102 and 103 synchronize, we will need password-less SSH access between them.

To do so, we need to first generate an SSH key. From robots that will connect to the master, we need to run this command in the Terminal:

ssh-keygen -t rsa -b 4096   # leave default locations and do not input a password (leave blank)

Then, we need to copy the key to the master or leader robot:

ssh-copy-id disrobot@192.168.2.101   # the system will ask for the password to access 101

After this, we should be able to access Robot 1 from other robots, by simply:

ssh disrobot@192.168.2.101   # no need for password

This task has been completed.

These instructions will guide us through using Supercollider via a Terminal window connected to our robots via SSH. For example, in a Macbook Pro, after doing ssh disrobot@192.168.2.101, we will be in our remote computer. We will need multiple windows. Towards the end we will try to create a script to open everything we need at once.

First, it is recommended to start jackd:

jackd -d alsa --device hw:S3 --rate 44100 --period 2048 &   # hw:S3 is the location of our SoundBlaster external sound card.

Then, we need to boot the server from another Terminal window:

scsynth -u 57110 &

Then, we need to launch Supercollider from another Terminal window, this is how we launch it:

QT_QPA_PLATFORM=offscreen sclang

From within the window that activated sclang, we need to connect to our already booted server:

s = Server.local;
s.boot;

Then we can finally use Supercollider from within the Terminal window. A simple test should confirm:

{ SinOsc.ar(440, 0, 0.2) }.play;

To stop:

s.freeAll

Knowing this, we now want to be able to run an entire .scd file from the terminal. Let’s see.

This is mainly for the RPLiDARS, considering that we will have two identical lidars for each robot, we need to be able to identify them. The way we will identify them is by the exact physical USB port they’re connected to. This way, if we run the command:

udevadm info -a -n /dev/ttyUSB0   # when only the top lidar is connected

We will see a lot of information, one of which will look like this:

KERNELS=="4-1:1.0"   # this refers to the physical location

For our top lidar, the location is on the top left corner of the USB ports. Then, we must create rules:

sudo nano /etc/udev/rules.d/99-lidar.rules   # this will create a new document the first time

Inside the document, we will add the symlink rules:

SUBSYSTEM=="tty", KERNELS=="4-1:1.0", SYMLINK+="top_lidar"

Then we close the rules document and, to have the rule be effective immediately, we must reload udev rules:

sudo udevadm control --reload-rules
sudo udevadm trigger

We can repeat the process for the bottom lidar, adding to the rules document a second line with:

SUBSYSTEM=="tty", KERNELS=="2-1:1.0", SYMLINK+="bottom_lidar"

The bottom lidar is connected to the bottom left USB port. Then reload udev rules, now both lidars will be accessible from /dev/top_lidar and /dev/bottom_lidar instead of /dev/ttyUSB0 and /dev/ttyUSB1.

This task is complete.

We need a button that acts as a lock and stops all movement and sound from the robot but doesn’t turn it off (hence not the on/off switch). We need to be able to press it to stop and press it again to resume.

We want to create a node that enables a «wandering» behavior. This means that our robot should move around space, avoiding obstacles without any specific target.

We will create a package to code this node using Python, as detailed here:

cd ~/<workspace_name_ws>/src
ros2 pkg create --build-type ament_python robot_wander

This will create a series of files and folders that, in some cases, we need to modify slightly. First, we will change ‘setup.py.’ Inside /src/setup.py, by default, the code looks like this:

from setuptools import find_packages, setup
package_name = 'robot_wander'
setup(
    name=package_name,
    version='0.0.0',
    packages=find_packages(exclude=['test']),
    data_files=[
        ('share/ament_index/resource_index/packages',
            ['resource/' + package_name]),
        ('share/' + package_name, ['package.xml']),
    ],
    install_requires=['setuptools'],
    zip_safe=True,
    maintainer='disrobot',
    maintainer_email='disrobot@todo.todo',
    description='TODO: Package description',
    license='TODO: License declaration',
    tests_require=['pytest'],
    entry_points={
        'console_scripts': [
        ],
    },
)

We need to change

We need to install ‘python-osc’ first:

pip install python-osc --break-system-packages   # we need to break system packages, but this won't hurt the system

Then,

Ticks measure encoder data; we need to know how many ticks are recorded per wheel rotation and how many ticks are recorded per meter traveled. The following table contains all of our data both for the (old) 313pm motors and the (new) 118rpm motors:

https://docs.google.com/spreadsheets/d/1gXIYVflYtpmY3lc_lc8FVnALPolJMXfjmJMj7WFF6CY/edit?usp=sharing

Make sure SSH access is password-less by following steps here.

tmuxinator-button.service

sudo rsync -avz --delete -e ssh disrobot@192.168.2.101:/etc/systemd/system/tmuxinator-button.service /etc/systemd/system/tmuxinator-button.service

Python Codes

rsync -avz --delete -e ssh disrobot@192.168.2.101:/home/disrobot/PythonCodes/ /home/disrobot/PythonCodes/

tmuxinator configs (Consolidated Launch Files)

rsync -avz --delete -e ssh disrobot@192.168.2.101:/home/disrobot/.config/tmuxinator/ /home/disrobot/.config/tmuxinator/

slam_ws (ROS2 workspace):

rsync -avz --delete -e ssh disrobot@192.168.2.101:/home/disrobot/slam_ws/ /home/disrobot/slam_ws/

SC (Supercollider folder)

rsync -avz --delete -e ssh disrobot@192.168.2.101:/home/disrobot/SC/ /home/disrobot/SC/

ROS2 LAUNCH COMMANDS

Consolidated Launch w/ Tmux | Tmuxinator

tmuxinator start DR1_session
tmuxinator start DRX_session

Individual Launch Commands

SSH Terminal Windows:

DR3:

ssh disrobot@192.168.0.53 #change IP address accordingly

Roboclaw Node:

cd ~/slam_ws
. install/local_setup.bash
ros2 launch roboclaw_node_ros roboclaw_node.launch.py

Twist Mux:

cd ~/slam_ws
. install/local_setup.bash
ros2 launch twist_mux twist_mux_launch.py config_topics:=/home/disrobot/slam_ws/src/twist_mux/config/twist_mux_topics.yaml

Robot State Publisher:

cd ~/slam_ws
. install/local_setup.bash
ros2 launch disrobot_description urdf_visualize.launch.py 

Cartographer:

cd ~/slam_ws
. install/local_setup.bash
ros2 launch cartographer_ros backpack_2d_custom.launch.py configuration_directory:=/home/disrobot/slam_ws/src/cartographer_ros/configuration_files configuration_basename:=backpack_2d_custom.lua

—or—

. install/local_setup.bash
ros2 launch cartographer_ros backpack_2d_custom.launch.py configuration_directory:=/home/disrobot/ros2_ws/src/cartographer_ros/configuration_files configuration_basename:=backpack_2d_custom.lua

—x—

Teleop:

cd ~/slam_ws
. install/local_setup.bash
ros2 run teleop_twist_keyboard teleop_twist_keyboard --ros-args -r /cmd_vel:=/key_vel  # correct remapping

Nav2:

cd ~/slam_ws
. install/local_setup.bash
ros2 launch nav2_bringup bringup_launch.py params_file:=/home/disrobot/slam_ws/src/nav2_bringup/params/nav2_params.yaml

—or—

cd ~/ros2_ws
. install/local_setup.bash
ros2 launch nav2_bringup bringup_launch.py params_file:=/home/disrobot/ros2_ws/src/nav2_bringup/params/nav2_params.yaml

—x—

————

In Remote Desktop:

*TOP* RPLiDAR Node:

sudo chmod 777 /dev/dri/renderD128
cd ~/slam_ws
. install/local_setup.bash
QT_QPA_PLATFORM=xcb ros2 launch rplidar_ros view_rplidar_a1_launch.py

—or—

cd ~/slam_ws
. install/local_setup.bash
ros2 launch rplidar_ros rplidar_a1_launch.py

*BOTTOM* RPLiDAR Node:

sudo chmod 777 /dev/dri/renderD128
cd ~/slam_ws
. install/local_setup.bash
QT_QPA_PLATFORM=xcb ros2 launch rplidar_ros view_rplidar_a1b_launch.py serial_port:=/dev/bottom_lidar frame_id:=bottom_laser remap_scan:=/bottom_scan

—or—

cd ~/slam_ws
. install/local_setup.bash
ros2 launch rplidar_ros rplidar_a1b_launch.py serial_port:=/dev/bottom_lidar frame_id:=bottom_laser remap_scan:=/bottom_scan

Joint State Publisher Gui

cd ~/slam_ws
. install/local_setup.bash
ros2 run joint_state_publisher_gui joint_state_publisher_gui

———

Robot Wander:

cd ~/slam_ws
. install/local_setup.bash
ros2 launch robot_wander startwandering.launch.py  # RPLiDAR, Twist_Mux, Teleop, Roboclaw < Necessary

ros2osc:

cd ~/slam_ws
. install/local_setup.bash
ros2 run ros2osc ros2osc_node

lift_routine:

cd ~/slam_ws
. install/local_setup.bash
ros2 launch lift_routine liftroutine.launch.py

timer_conductor:

cd ~/slam_ws
. install/local_setup.bash
ros2 launch timer_conductor timer.launch.py

Synchronization is an essential aspect of creating coordinated movement. We must be able to simultaneously send a command to all robots, making them as synchronized as possible.

SOLUTION

By having all robots in the same network (see this post), we can take advantage of the ROS2 middleware’s Data Distribution Service (DDS) features, which distribute data (topics) not only within the individual system running ROS2 nodes but also to other systems running nodes within the same network.

However, we must be careful to keep topics private from nodes not meant to be shared. An example of a topic that has to stay within its domain is /cmd_vel (the topic that sends movement instructions to the motor controller). This topic needs to remain private. Otherwise, the movement command from one robot could end up moving another robot in the same network, causing conflicting instructions and potential damage.

We can have a distinct Domain ID for each robot to avoid topic conflicts. Robot 1 can have Domain ID 1. Robot 2 can have Domain ID 2, and so on. Topics within a shared domain ID can be published and subscribed through the network, but topics within different domain IDs can’t.

For each robot, we must first ensure the Domain ID is distinct:

echo $ROS_DOMAIN_ID

If there is no result or if the Domain ID returned is not the desired one, we need to change it:

export ROS_DOMAIN_ID=<your_domain_id> # For example, 1.

Finally, if we want to save this setting so that every shell (Terminal) session defaults to the Domain ID that we want, we need to do the following:

echo "export ROS_DOMAIN_ID=<your_domain_id>" >> ~/.bashrc

Upon opening new shell sessions, «echo $ROS_DOMAIN_ID» will return the number we inscribed as the default. This will ensure that each robot is operating, for the most part, with all topics and nodes within its domain.

Now that we have separated each domain to avoid conflicts, we can run specific nodes within different Domain IDs. All we need to do is open a new shell session where, before running our nodes, we will re-define the Domain ID for only that single shell session (or Terminal window). So if our robot is operating in Domain ID = 3, but we want to run specific nodes that synchronize with other robots (for example, another robot operating in Domain ID=1), then we can, for that single shell session:

export ROS_DOMAIN_ID=1

If we were to publish a topic from within this window after defining Domain ID=1, for example:

. install/local_setup.bash
ros2 topic pub /robot_topic std_msgs/String "data: 'Hello from Robot 3'"

Then in robot 1, we will be able to find /robot_topic:

. install/local_setup.bash

ros2 topic list # this will show the list of topics currently available, one of which should be /robot_topic

ros2 topic echo /robot_topic. # this will publish the message received from robot 3.

We can see that

We need to use two networks to have a ubiquitous network solution for our robots. The first one is to connect to any local Wi-Fi networks available. For example, the DXARTS Gallery has its network, while the DXARTS Electronics has another one. The University of Washington has its network, too, and so on. However, the connection between remote and local computers to access Ubuntu remotely should occur from within our router. This will provide a stable solution that can be taken, together with our router, anywhere we want instead of depending on the local Wi-Fi.

We need to be able to connect to two Wi-Fi networks simultaneously to have both internet access and a local network.

SOLUTION

We have purchased a TP-LINK Archer A6 Wireless Router:

• TP-Link AC1200 Gigabit WiFi Router (Archer A6)

With our new router, we created the «disobedientrobots» Wi-Fi network. When we first turned our router on, we accessed the configuration pages at 192.168.0.1. After creating a password for the first time, we changed the network name to be «disobedientrobots» and the password «disobedientrobots» too. Then, we accessed the LAN settings by clicking on Advanced at the top of the interface and then, from the sidebar, selecting Network > LAN.

• Change the current LAN IP address from 192.168.0.1 to 192.168.2.1. By using 192.168.2.1, we’re making sure that we don’t conflict with the default subnet many routers usually pick (192.168.0.x).

Then, to access two Wi-Fi networks using a single Raspberry Pi, we need to use an additional USB Wi-Fi adapter. For that purpose, we are using:

• Geekworm NVIDIA Jetson Nano Dual Band Wireless USB 3.0 WiFi Adapter 5GHz+2.4GHz 1200M

This adapter is «plug-and-play» with our Raspberry Pi 5’s. As soon as it is connected via USB to the RPi, we can connect to 2 separate internet connections, one with the internal Wi-Fi antenna and one with our newly inserted USB adapter antenna.

• Using the internal Wi-Fi antenna, we will connect to the no-internet «disobedientrobots» network. The computer will be assigned an IP Address within the subnet range, such as 192.168.2.101.
• We will then enter the configuration settings for this network, and under the IPv4 tab, we will select «manual» instead of «automatic.» This will allow us to assign a static IP to each robot manually instead of automatically assigning IP addresses. For the Address field, we will use the computer’s assigned IP address, for example, 192.168.2.101. For the Netmask, we will use 255.255.255.0; for the Gateway, we will use the router’s IP address, 192.168.2.1.
• We will connect to the internet network, in our case, «DX-STUDIO» or «DX-GALLERY,» with the external USB adapter Wi-Fi antenna.
• NOTE: If we do this the other way around, attempting to connect to our RPis via RDP remotely will fail. This could be because Ubuntu has the routing internally assigned to the internal Wi-Fi card.

Once connected to both networks, we can access these computers from within the network. From a Mac OS, we won’t be able to connect to two separate networks using an USB adapter antenna. However, by using a virtual machine (the UTM app) running Ubuntu within my MacBook Pro, I was able to access the internet using the USB adapter antenna while the main Mac OS Wi-Fi connection was linked to «disobedientrobots». It is a workaround to be able to have internet access although it happens through a Virtual Machine.

Now we can use software like Royal TSX to connect to our robots using IP Addresses from the 192.168.2.x subnet. The network is stable, and we are able to move around our own router.

For now, this problem is solved.

When trying to install packages to use the GPIO pins, it was found that there is an issue with installing RPi.GPIO, but there is an alternative package called gpiozero. This has almost all the functionalities required to get the lift moving with the relay and motor controller. First, we need to install gpiozero:

sudo apt-get install python3-gpiozero

Always use (replace … with import you want to use):

from gpiozero import …

Using example codes and combining them, a test code to get the GPIO pins to work with the relay and motor controller was made.

from gpiozero import PWMOutputDevice, DigitalOutputDevice, OutputDevice
from time import sleep

# Define pins
REL_PIN = 25
PWM_PIN = 12
DIR_PIN = 16

# Initialize PWM and direction pin
motor_pwm = PWMOutputDevice(PWM_PIN, frequency=1000)
motor_dir = DigitalOutputDevice(DIR_PIN)
device = OutputDevice(REL_PIN)

# Function to set motor direction
def set_motor_direction(forward=True):
    motor_dir.value = 0 if forward else 1

try:
    # Run motor in the opposite direction
    device.on() # Turns the relay on    
    set_motor_direction(forward=False)
    motor_pwm.value = 0.5  # 50% duty cycle
    print("Motor moving down at 50% speed")
    sleep(10)

    # Stop motor
    motor_pwm.value = 0
    print("Motor stopped")
    device.off() # Turns the relay off
    sleep(1)

    for i in range(1,10):
        device.on()
        # Run motor in one direction
        set_motor_direction(forward=True)
        motor_pwm.value = (i/10) # duty cycle
        print("Motor moving up at", i/10)
        sleep(5)

        motor_pwm.value = 0
        print("Motor stopped")
        device.off()
        sleep(1)

        device.on()
        set_motor_direction(forward=False)
        motor_pwm.value = (i/10) # duty cycle
        print("Motor moving down at", i/10)
        sleep(5)

finally:
    # Ensure motor is stopped
    motor_pwm.value = 0
    print("Motor stopped")
    device.off()

Started off connecting motor controller to lift; the connection can be visualized using the electrical diagram in Hardware section. The DIR, PWM, and GND pins on the motor controller are connected to the GPIO pins on the Raspberry Pi, while the Motor A and Motor B pins are connected to the motor on the lift.

The power to the motor controller is connected to a relay, which will turn on the motor controller only when the lift is needed. This is to keep the motor controller safe since it produces too much heat in idle state. The relay is then connected to a 24V power supply.

First, we need to install Cartographer for ROS2:

sudo apt update
sudo apt install ros-jazzy-cartographer
sudo apt install ros-jazzy-cartographer-ros

Then, we need to set open permissions for the files in both cartographer packages. Otherwise, we will encounter permission problems:

sudo chmod -R 777 /opt/ros/jazzy/share/cartographer/
sudo chmod -R 777 /opt/ros/jazzy/share/cartographer_ros/

We are first going to map using the backpack_2d.launch.py launch file. However, we will create a custom version of the same launch file for three reasons. The first one is that we don’t need everything that is being launched with that launch file. Second, we must create parameters to modify with the launch command. Third, we need to have a launch file and parameter files that aren’t affected by future ROS2 upgrades, which could reset modifications to the original files.

cd /opt/ros/jazzy/share/cartographer_ros/launch/
touch backpack_2d_custom.launch.py  # this creates the file

Then, we add the following code to the launch file:

"""
  Copyright 2018 The Cartographer Authors
  Copyright 2022 Wyca Robotics (for the ros2 conversion)
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at
       http://www.apache.org/licenses/LICENSE-2.0
  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License.
"""
from launch import LaunchDescription
from launch.actions import DeclareLaunchArgument, IncludeLaunchDescription
from launch.conditions import IfCondition, UnlessCondition
from launch.substitutions import LaunchConfiguration
from launch_ros.actions import Node, SetRemap
from launch_ros.substitutions import FindPackageShare
from launch.launch_description_sources import PythonLaunchDescriptionSource
import os
def generate_launch_description():
    ## ***** Launch arguments *****
    use_sim_time_arg = DeclareLaunchArgument('use_sim_time', default_value = 'False')
    config_dir_arg = DeclareLaunchArgument(
        'configuration_directory',
        default_value=os.path.join(FindPackageShare('cartographer_ros').find('cartographer_ros'), 'configuration_files'),
        description='Configuration directory for Cartographer')
    config_basename_arg = DeclareLaunchArgument(
        'configuration_basename',
        default_value='backpack_2d.lua',
        description='Configuration file for Cartographer Backpack 2d')
    ## ***** File paths ******
    pkg_share = FindPackageShare('cartographer_ros').find('cartographer_ros')
    # urdf_dir = os.path.join(pkg_share, 'urdf')
    # urdf_file = os.path.join(urdf_dir, 'backpack_2d.urdf')
    # with open(urdf_file, 'r') as infp:
    #    robot_desc = infp.read()
    ## ***** Nodes *****
    # robot_state_publisher_node = Node(
    #    package = 'robot_state_publisher',
    #    executable = 'robot_state_publisher',
    #    parameters=[
    #        {'robot_description': robot_desc},
    #        {'use_sim_time': LaunchConfiguration('use_sim_time')}],
    #    output = 'screen'
    #    )
    cartographer_node = Node(
        package = 'cartographer_ros',
        executable = 'cartographer_node',
        parameters = [{'use_sim_time': LaunchConfiguration('use_sim_time')}],
        arguments = [
            '-configuration_directory', LaunchConfiguration('configuration_directory'),
            '-configuration_basename', LaunchConfiguration('configuration_basename')],
        remappings = [
            ('echoes', 'horizontal_laser_2d')],
        output = 'screen'
        )
    cartographer_occupancy_grid_node = Node(
        package = 'cartographer_ros',
        executable = 'cartographer_occupancy_grid_node',
        parameters = [
            {'use_sim_time': True},
            {'resolution': 0.05}],
        )
    return LaunchDescription([
        use_sim_time_arg,
        config_dir_arg,
        config_basename_arg,
        # Nodes
        # robot_state_publisher_node,
        cartographer_node,
        cartographer_occupancy_grid_node,
    ])

Finally, we create a configurations_files folder within our own workspace:

sudo mkdir /home/disrobot/ros2_ws/src/cartographer_ros/ /home/disrobot/ros2_ws/src/cartographer_ros/configuration_files/

We copy the contents of the ROS2 installation Cartographer Configuration Files onto our own ros2_ws workspace:

sudo cp -r /opt/ros/jazzy/share/cartographer_ros/configuration_files/ /home/disrobot/ros2_ws/src/cartographer_ros/

Then, we edit permissions to make sure they’re set correctly:

sudo chown -R disrobot:disrobot /home/disrobot/ros2_ws/src/cartographer_ros/

Finally, we create our custom configuration file:

cd ~/ros2_ws/src/cartographer_ws/configuration_files/
touch backpack_2d_custom.lua

Using an editor, we paste the following code into backpack_2d_custom.lua:

-- Copyright 2016 The Cartographer Authors
--
-- Licensed under the Apache License, Version 2.0 (the "License");
-- you may not use this file except in compliance with the License.
-- You may obtain a copy of the License at
--
--      http://www.apache.org/licenses/LICENSE-2.0
--
-- Unless required by applicable law or agreed to in writing, software
-- distributed under the License is distributed on an "AS IS" BASIS,
-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-- See the License for the specific language governing permissions and
-- limitations under the License.
include "map_builder.lua"
include "trajectory_builder.lua"
options = {
  map_builder = MAP_BUILDER,
  trajectory_builder = TRAJECTORY_BUILDER,
  map_frame = "map",
  tracking_frame = "laser",
  published_frame = "base_link",
  odom_frame = "odom",
  provide_odom_frame =false,
  publish_frame_projected_to_2d = true,
  use_pose_extrapolator = true,
  use_odometry =false,
  use_nav_sat = false,
  use_landmarks = false,
  num_laser_scans = 1,
  num_multi_echo_laser_scans = 0,
  num_subdivisions_per_laser_scan = 1,
  num_point_clouds = 0,
  lookup_transform_timeout_sec = 0.2,
  submap_publish_period_sec = 0.3,
  pose_publish_period_sec = 5e-3,
  trajectory_publish_period_sec = 30e-3,
  rangefinder_sampling_ratio = 1.,
  odometry_sampling_ratio = 1.,
  fixed_frame_pose_sampling_ratio = 1.,
  imu_sampling_ratio = 1.,
  landmarks_sampling_ratio = 1.,
}
MAP_BUILDER.use_trajectory_builder_2d = true
TRAJECTORY_BUILDER_2D.num_accumulated_range_data = 0.1
TRAJECTORY_BUILDER_2D.max_range = 3.5
TRAJECTORY_BUILDER_2D.missing_data_ray_length = 3.
TRAJECTORY_BUILDER_2D.use_imu_data = false
TRAJECTORY_BUILDER_2D.use_online_correlative_scan_matching = true 
TRAJECTORY_BUILDER_2D.motion_filter.max_angle_radians = math.rad(0.1)
POSE_GRAPH.constraint_builder.min_score = 0.65
POSE_GRAPH.constraint_builder.global_localization_min_score = 0.7
return options

Finally, when we run our Cartographer launch file, we will use the following command:

cd ~/ros2_ws
. install/local_setup.bash
ros2 launch cartographer_ros backpack_2d_custom.launch.py configuration_directory:=/home/disrobot/ros2_ws/src/cartographer_ros/configuration_files configuration_basename:=backpack_2d_custom.lua

We should be able to run to have a map now.

We need to play audio coming directly out of the robot while accessing from the remote login (RDP) option.

First, we need to open alsa-state.service, found in /usr/lib/systemd/system/alsa-state.service.

Then, only add «!» as «=!/etc/alsa/state-daemon.conf».

This should allow the audio system to start without a problem. As described in a post below, the Sound Blaster card needs to be selected from the qjackctl setup menu at some point.

This task has been completed.

Remotely:

sshpass -p '*********' autossh -R 4004:localhost:22 disobedientrobot@209.42.195.109 -p 2200

Locally:

ssh -i '/Users/nka/Desktop/DXARTS Projects/SSH Keys/DR2024_local' -L 4004:localhost:4004 disobedientrobot@209.42.195.109 -p 2200

ssh -i '~/DR2024_local' -L 4004:localhost:4004 disobedientrobot@209.42.195.109 -p 2200   # UTM Ubuntu DR4

Then, locally:

ssh -p 4004 disrobot@localhost

Then we are connected via SSH in terminal to our remote computer. We need to add the remote command to initialize automatically at start-up. TBD.

Upon running RViZ2 (remotely), I encountered the following errors:

~/ros2_ws$ QT_QPA_PLATFORM=xcb rviz2   # run rviz2 remotely
TU: error: ../src/freedreno/vulkan/tu_knl.cc:208: failed to open device /dev/dri/renderD128 (VK_ERROR_INCOMPATIBLE_DRIVER)
Opening /dev/dri/renderD128 failed: Permission denied
MESA: error: ZINK: vkEnumeratePhysicalDevices failed (VK_ERROR_INITIALIZATION_FAILED)
MESA: error: ZINK: failed to choose pdev
glx: failed to create drisw screen
TU: error: ../src/freedreno/vulkan/tu_knl.cc:208: failed to open device /dev/dri/renderD128 (VK_ERROR_INCOMPATIBLE_DRIVER)
Opening /dev/dri/renderD128 failed: Permission denied
MESA: error: ZINK: vkEnumeratePhysicalDevices failed (VK_ERROR_INITIALIZATION_FAILED)
MESA: error: ZINK: failed to choose pdev
glx: failed to create drisw screen
[INFO] [1722121592.924541298] [rviz2]: Stereo is NOT SUPPORTED
[INFO] [1722121592.924652502] [rviz2]: OpenGl version: 4.5 (GLSL 4.5)
[INFO] [1722121593.003604851] [rviz2]: Stereo is NOT SUPPORTED

To fix them, I added the user to the video group:

sudo usermod -aG video disrobot

Then, I installed Vulkan:

sudo apt install vulkan-tools   # alternatively vulkan-utils (untested)

Finally, I installed mesa drivers:

sudo apt install mesa-utils

This resulted in an RViZ2 launch with no errors, except for «Stereo is NOT SUPPORTED» which is more of a warning.

This task has been completed.

To prevent us from having to run the following command:

sudo chmod 777 /dev/ttyACM0

Whenever a program needs to access anything connected to our USB ports, we can add our user ‘disrobot’ to the ‘dialout’ group. To see in which groups our user participates, we can run the following command:

groups disrobot

To add ‘disrobot’ to the ‘dialout’ group, which gives permanent permission/access to serial ports, we run:

sudo usermod -aG dialout disrobot   # change 'disrobot' for your username if it is different.

Afterwards, log out and log back in. Now, there should not be a need to give permissions every time we need to access devices.

This task has been completed.

We acquired a Shelly Plus 1 remote switch and followed the instructions in this video (the video explains it for 12V DC power) to attach it to our 24V power supply. We can now power on/off our robot (only DR2 for now) from anywhere in the world with an internet connection.

This task has been completed.