Contents
What we're building is illustrated by the diagram above.
- A 4 rPi stack
- A WiFi to Ethernet Bridge
- nginx with reverse proxy to each rPi
Physically, it looks like this:
The idea is:
- Given domain names (or in this article, a public IP:port to your router)...
- Route the TCP/IP packets to the top rPi over WiFi...
- Which then nginx routes to any of the 4 rPi's over Ethernet...
- Via "DNS Masquerade" which is used to implement a WiFi to Ethernet bridge.
We use DNS Masquerade to create a WiFi to Ethernet bridge. The IP addresses of the physically wired rPi's will be 10.1.1.1 through 1.1.1.4. Two blog posts were used to figure out how to do this:
It seems that since 2017, the way dnsmasq is configured has changed a bit. Most of the steps (except creating a static wireless IP, which broke my wireless on the rPI) regarding the initial setup were fine. For the rest of the dnsmasq steps (specifically the config files), I had to follow the steps in the 2018 blog post.
Execute these commands to get everything up to date and install dnsmasq.
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install dnsmasq
Verify that your WiFi is set up. This file should have contents:
sudo nano /etc/wpa_supplicant/wpa_supplicant.conf
If not, run raspi-config, even though your rPi may already be set up to use Wifi, and configure your WiFi (probably again) from the wireless configuration menu option.
Edit: sudo nano /etc/sysctl.conf and uncomment (remove the '#') the line #net.ipv4.ip_forward=1
Run these commands:
sudo iptables -t nat -A POSTROUTING -o wlan0 -j MASQUERADE
sudo iptables -A FORWARD -i wlan0 -o eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT
sudo iptables -A FORWARD -i eth0 -o wlan0 -j ACCEPT
sudo sh -c "iptables-save > /etc/iptables.ipv4.nat"
The above commands (those that I can figure out and should be obvious) forward wlan0 to eth0, and eth0 to wlan0. The configuration file is then saved and with the following command, restored when the rPi boots:
Edit sudo nano /etc/rc.local and just above the exit 0, add the line iptables-restore < /etc/iptables.ipv4.nat.
Create /etc/network/interfaces.d/eth0 with the contents:
auto eth0
allow-hotplug eth0
iface eth0 inet static
address 10.1.1.1
netmask 255.255.255.0
gateway 10.1.1.1
Create /etc/dnsmasq.d/bridge.conf with the contents:
interface=eth0
bind-interfaces
server=8.8.8.8
domain-needed
bogus-priv
dhcp-range=10.1.1.2,10.1.1.254,12h
Reboot. Try out these commands:
ifconfig
You should see (your wireless IP will most likely be different):
eth0: flags=4163<up,broadcast,running,multicast> mtu 1500
inet 10.1.1.1 netmask 255.255.255.0 broadcast 10.1.1.255
...
wlan0: flags=4163<up,broadcast,running,multicast> mtu 1500
inet 192.168.0.15 netmask 255.255.255.0 broadcast 192.168.0.255
</up,broadcast,running,multicast></up,broadcast,running,multicast>
Also execute:
sudo iptables -L
You should see:
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
ACCEPT all -- anywhere anywhere
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
If not, review the steps above and the article links. If all looks good, you can connect a device to the ethernet hub.
Laptop Wifi:
Laptop connecting to rPi Wifi-Ethernet bridge:
The odd thing about this is when I ran the test the previous evening, the rPi Wifi-Ethernet bridge was faster. Go figure.
These steps should be performed for each rPi as it makes working to them much easier than hooking up four monitors, keyboards, and mice, or moving cables back and forth all the time.
Use raspi-config to enable SSH for each rPI:
sudo raspi-config
which brings up a simple UI:
Navigate to item 5 "Interface Options"
Select "P2 SSH - Enable/Disable remote command line access to your Pi using SSH"
Select "Yes" when prompted "Would you like the SSH server to be enabled?"
Select "OK"
Select "Finish" to exit the config app.
Read more about the raspi-config app here.
I'll be doing development with .NET Core and C#, so let's install this next on all four rPI's:
wget <a href="https://download.visualstudio.microsoft.com/download/pr/
36bff52b-2cdd-4011-8e92-d00f7537704f/9885ba267b1ef29a1401adc387d9a016/
dotnet-sdk-2.2.101-linux-arm.tar.gz">https://download.visualstudio.microsoft.com/
download/pr/36bff52b-2cdd-4011-8e92-d00f7537704f/9885ba267b1ef29a1401adc387d9a016/
dotnet-sdk-2.2.101-linux-arm.tar.gz</a>
sudo mkdir -p /bin/dotnet && sudo tar zxf dotnet-sdk-2.2.101-linux-arm.tar.gz -C /bin/dotnet
export DOTNET_ROOT=/bin/dotnet
export PATH=$PATH:/bin/dotnet
sudo ln -s /bin/dotnet/dotnet /usr/local/bin
This:
- Installs dotnet at /bin/dotnet
- sets up paths
- creates a symlink (shortcut) called "
dotnet"
We can now use the wifi-ethernet link to set up the other rPI's without enabling Wifi on them!
I'd like my rPI's to have static IPs. The top one is 10.1.1.1, so it seems logical that the next three should be .2 through .4. This is easily done by editing /etc/dhcpcd.conf and finding the following section, uncommenting the configuration and setting the IPv4 addresses accordingly:
# Example static IP configuration:
interface eth0
static ip_address=10.1.1.2/24
static ip6_address=fd51:42f8:caae:d92e::ff/64
static routers=10.1.1.1
static domain_name_servers=10.1.1.1 8.8.8.8 fd51:42f8:caae:d92e::1
Reboot, run ifconfig, and verify the static IP address:
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.1.1.2 netmask 255.255.255.0 broadcast 10.1.1.255
The following pieces I already wrote about in more detail here, so this is just a quick "do this" set of steps. These steps are performed on the whatever rPI (in my case, the top one) is set up as the WiFi-Ethernet bridge.
Install PuTTY on 10.1.1.1, so we can talk with the other rPI's on .1, .2, .3, and .4 (yeah, do this for .1 as well, even though this is under the section Setup the other 3 rPI's.
sudo apt-get install -y putty
I'm going to install nginx on .1 with the intention that it will route the public WiFi address ports to specific rPI's that are responsible for serving the web site -- One website per rPI.
Install nginx:
sudo apt-get install nginx
Auto-start nginx at boot:
cd /etc/init.d
sudo update-rc.d nginx defaults
Reboot and open the Chromium browser on the rPI desktop, or from your laptop connected via Ethernet to the hub, and navigate to http://10.1.1.1.
Next, we want to port forward the Wifi address to each of the Ethernet addresses. I'm going to configure my router like this:
3001-3004 are the inbound ports, 8081-8084 are the ports we want nginx to intercept and route to 10.1.1.1-4. This is really just for testing. In "the real world", I would have set up one or more domain names and the router would route port 80 (HTTP) and port 443 (HTTPS) to the rPi that is the WiFi-Ethernet bridge, using an authorized SSL certificate and SNI to route the domain(s) to the appropriate rPI. nginx can also route specific URL paths to different rPI's, so one could create a setup where different pages are handled by different rPI's, but that's beyond the scope of this "proof of concept" article.
Given the router configuration above, each port exposed to the public is routed to a specific rPI. This is the route declaration that gets added to the nginx /etc/nginx/sites-available/default file:
server {
listen 8081;
location / {
proxy_pass http://10.1.1.1:8080;
}
}
server {
listen 8082;
location / {
proxy_pass http://10.1.1.2:8080;
}
}
server {
listen 8083;
location / {
proxy_pass http://10.1.1.3:8080;
}
}
server {
listen 8084;
location / {
proxy_pass http://10.1.1.4:8080;
}
}
Reload nginx with:
sudo /etc/init.d/nginx reload
Test this by installing dotnet-serve on one of the rPI's (I chose the 4th one):
dotnet tool install --global dotnet-serve
Then run:
export PATH="$PATH:/home/pi/.dotnet/tools"
export DOTNET_ROOT=/bin/dotnet
dotnet-serve -a 10.1.1.[rPI IP address]
The exports are necessary to find dotnet-serve and for dotnet-serve to find dependant libraries.
For example, the 4th rPI in my stack, the final command is:
dotnet-serve -a 10.1.1.4
For some reason, I get an access denied when I try to specify port 80, which is why nginx was configured above with port 8080.
Find your public IP and type in:
http://[your public ip]:3004
You should get back:
and dotnet-serve should be displaying:
You can navigate the folders and download the MagPi PDF!
In my article, Slack Chatting with your rPi, I used Slack to communicate to a single rPi. We can use Slack to issue commands to all the rPi's in the stack. This is really useful for doing things like shutting down or rebooting all the rPi's!
Once you've WinSCPs, the slackbot\bin\Debug\netcoreapp2.2\linux-arm\publish folder contents over to each rPi, change the slackBot file to an executable (this only has to be done once) and run it:
chmod +755 ./slackBot
./slackBot
Do this for each rPi. Now, ping the rpi on your Slack channel:
All four of them responded!
Let's look at the memory utilization of each rPi:
Of course, we'd like to know which rPi is reporting, as they may not necessarily be in order. We can report the IP address whenever the rPi responds by using a bash script executed in the C# slackBot application to get the eth0 address:
static void GetIPAddress()
{
ipAddress = "ifconfig eth0 | grep \"inet \"".Bash().Trim().
RightOf("inet").LeftOf("netmask").Trim();
Console.WriteLine($"IP: {ipAddress}");
}
We can then prepend this address to everything sent over the Slack channel by the rPi:
...
if (router.TryGetValue(cmd, out Func<string, List<string>, string> fnc))
{
ret = ipAddress + ": " + fnc(data, options);
}
else
{
// Try as bash command.
string cmdline = (cmd + " " + data).Trim();
ret = $"```\r\n{ipAddress}\r\n{cmdline.Bash()}```";
}
...
The results are nice:
or, for bash commands, here's an example output with the rPi's IP address:
Of course, it would also be nice to talk to a specific rPi. We'll do this by adding the last digit of the rPi's IP address as an option (using the "--" syntax) to talk to just one of them. In the message receive handler, we check for a --[ddd] where [ddd] is the digits in the last part of the IP address:
...
while (cmd.StartsWith("--"))
{
var opt = cmd.LeftOf(" ");
// Check if a specific address [n] in our IP x.y.z.n is specified.
// If that's not our address, just exit now.
if (opt.Length > 2 && char.IsDigit(opt[2]))
{
if (opt.Substring(2) != ip4thDigits)
{
return;
}
}
options.Add(opt);
cmd = message.text.RightOf(opt).Trim().LeftOf(" ");
}
Note the difference now when we use the --[ddd] option:
This is very cool! Everything is doing its thing:
- The router is routing
- The WiFi-ethernet bridge is bridging
- nginx is reverse proxy'ing
- the server is serving!
We also modified my Slack Chat application so that we can:
- Broadcast commands to all rPi's
- Send commands to individual rPi's
While there's a lot more to get done in terms of setting up domain names, SSL certs, and ngnix with SNI, the purpose of this article is how you take the first step in setting up an rPI stack, from which we can now serve pages, provide "compute" capability, play around with distributed data, etc.
Time to take a break.
These notes are for .NET Core 2.0 and I put them here because they are less obtuse (I suppose) than that "wget" command, and may be useful for when .NET Core 3.0 comes of age.
From: https://blogs.msdn.microsoft.com/david/2017/07/20/setting_up_raspian_and_dotnet_core_2_0_on_a_raspberry_pi/
- Run sudo apt-get install curl libunwind8 gettext. This will use the apt-get package manager to install three prerequiste packages.
- Run curl -sSL -o dotnet.tar.gz https://dotnetcli.blob.core.windows.net/dotnet/Runtime/release/2.0.0/dotnet-runtime-latest-linux-arm.tar.gz to download the latest .NET Core Runtime for ARM32. This is refereed to as armhf on the Daily Builds page.
- Run sudo mkdir -p /opt/dotnet && sudo tar zxf dotnet.tar.gz -C /opt/dotnet to create a destination folder and extract the downloaded package into it.
- Run sudo ln -s /opt/dotnet/dotnet /usr/local/bin` to set up a symbolic link...a shortcut to you Windows folks 😉 to the dotnet executable.
- Test the installation by typing
dotnet --help.
