Understanding USB Serial on Arduino Uno (There Is No Magic)

(source: http://gordonscruton.blogspot.com/2011/06/no-magic-please-part-1-learning.html)

A professor I had was fond of saying that the first thing to learn about computers is that there is no magic. From time to time, I am reminded of that truth. So let me tell you a story.

I've been playing around with a lighting project using some Supernight LED strips (unlike NeoPixels, these light strips allow you to control the hue but don't allow for individual LED clusters to be controlled; the entire strip takes on one color). After having some fun with them using the packaged-in controller, I decided to wire up my own based on some previous work and a tutorial from Make on how to control these lights (they're basically four-channel: one channel is +12v, and the other three can accept a PWM signal for color intensity). I used an Arduino UNO as the driver for the light control circuit. Everything was going well: I did a breadboard test, wired it all up, and put it in a nice project enclosure box.

To control the lights, I'm running a web service on a Raspberry Pi that communicates via USB serial to the Aruduino (in theory, the Pi could send the PWM signals directly, but I'm squeamish about wiring my harder-to-replace Pi to a 12 volt signal). I wrote a small state machine to accept some serial signals on the Arduino, and a small go service to take web requests and translate them into opening the serial port, sending the data, and closing the port.

When it works, it works great!

Trouble is, it didn't work. I could see my server sending the signals, and it believed they were sent correctly, but the Arduino wouldn't respond. I could even go into the Serial Monitor in the Arduino IDE and send the characters individually; they'd trigger the lights, but when I sent them from my server, the lights would go out. I couldn't debug by printing messages back over the serial connection, because I was using it to control the board (PROTIP: always add a couple LEDs to your enclosure; you never know when you're going to need the ability to just get a simple status signal from your Arduino without using serial).

The breakthrough came from adding a bit of code to the setup() function that caused the lights to quickly pulse red-green-blue to indicate setup was complete. I quickly learned that every time I tried to send a signal to change the light color from the Pi, the lights were pulsing---the Arduino was doing a full reboot. Why?

I had assumed that the Arduino was (at the hardware level) talking directly to the USB, and the Serial library I used emulated a serial connection in software. In reality, that's not the case at all. The secret is described clearly (well, clearly enough ;) ) in the wiring diagram for the UNO itself.

(Source)

The main processor is an ATMega 328P, but there's an entire second processor on the chip---an ATMega16U2 that sits between the main processor and the USB hardware. That processor has (normally) non-modifiable firmware that translates the USB serial protocol to a traditional 5v serial logic protocol. It also has a useful feature, sitting right in the middle of the diagram:

"There's yer problem."

It's a bit hard to tell in all the visual noise, but that's pin 13 on the USB controller wired to RESET on the main processor. So when does the USB controller reset the main processor? When a USB connection is initiated, the controller sends a DTR signal on pin 13. This signal reboots the processor. Every time my web service opened the serial device in Linux, it was causing the USB controller to trigger a DTR---the subsequently-sent bytes were then either being consumed or dropped on the floor. Fixing this was as simple as re-coding my service to open the device once and hold onto it for the entire life of the process. No sweat!

So this still leaves us the question: why is DTR wired to RESET? Here's the magic I was missing: The Arduino UNO uses the USB connection for two things: sending and receiving serial communications, and programming the Arduino itself. How does it know that the signal coming in is supposed to be new program logic and not serial commands for the program to consume? It turns out that the IDE reprograms the Arduino by starting a serial connection (causing DTR to pull the RESET pin). The first thing the Arduino's main processor does on reset is run firmware that listens for a few milliseconds for a sequence of bytes indicating new code incoming (these bytes are documented in the STK500 communication protocol v1)---if it receives those bytes, it writes them to flash memory, otherwise it gives up and begins running userland code. The Arduino Serial Monitor triggers the reset also, but doesn't send the new-code bytes. Previous to this feature being added, reprogramming the Arduino required the developer to time resetting the Arduino and starting the code upload somewhat precisely, which was annoying. Incidentally, if you own an Arduino and don't want this feature enabled, there are a couple of convenient ways to disable it so that your processor won't reboot every time it starts a serial connection.

So there you go: no magic, just a pair of processors with some interwoven activation logic that allows the same wire to serve as both communications and as the channel to download new program versions. Which, when it all operates smoothly, is pretty magic!

Personal Experience: Making a game-show buzzer controller

To kick off the new year, I teamed up with Aaron (he has a blog!) to put together a game show buzzer system for a project he's working on. We followed the directions from an Instructables guide, which were pretty straightforward. You'll want to read that all the way through if you want to follow in detail what we did; I'm going to talk about the highlights and gotchas we discovered, but not do a blow-by-blow replay of the Insructable itself.

Experience building the RadioShack 3D LED Cube Kit

While looking for something to use a gift card on, I stumbled across the Radio Shack 3D LED Cube Kit for Arduino. It'd been awhile since I'd tried building an electronics board project from scratch, and I'd seen some clever demos of "3D pixel" displays; I thought I'd try my hand at putting this together. What follows is my experience with it.

Return of the Arduino part 3: mount the transmitter LED and load in the control code.

With the IR sensor and the status indicator LEDs in place, the only remaining steps are to wire up the IR transmitter and load the control software. Relative to everything else, this is the easy part. :)

My IR LED of choice is a 940nm LED that I picked up from Radio Shack. I used a basically direct connection to one of the digital output ports on the Arduino. The only complexity of this part is  the desire to make the IR trasmitter pretty mobile; I need to be able to position the receiver and transmitter separately from each other so I don't need to put a table in front of the TV to hold the enclosure. Since the LED is external to the enclosure and positionable, I grabbed some male-female servo extension cables from Amazon. I snipped one in half and glued the male end to the enclosure, making an external-facing plug.

I then did away with the white wire (leaving only two channels) and soldered the black wire to the common ground used by the LEDs (the red wire gets a breadboard jumper wire and connected to digital port 13.

The LED side of the wiring is simpler. Cutting the male plug off of another servo extender and stripping out its white wire, I soldered the IR LED onto the remaining wires and added a bit of shrink tubing to clean everything up.

The final software package is available on Github. It's readily configurable for details, but its default configuration is as follows:

• Listen for a signal from my remote (I happen to have a spare "blue button" that works just great).
• When the signal is recognized, blink twice and begin a half-hour countdown.
• At end of countdown, fire the off signal and go back to listenening for input.

 I ran a test (with the timer temporarily overridden from a half-hour to five seconds), and I'm pretty happy with the results!

Overall, this has been a pretty educational project. In the future, I'll try and add the Ethernet shield so that I can wire computer control into the entertainment system.

Return of the Arduino part 2: status indicators are your friends

The essence of user-friendliness is feedback. I won't be rivaling Apple with my little creation, but having more than zero indication that the IR receiver is actually picking up signal is probably wise. So the next step in the project was to mount some LEDs to the case to give me a bit of status indication.

A quick trip to Radio Shack netted me three red LEDs (2.6v, 28mA, 10mcd), one green LED (2.1v, 25mA, 6.3mcd), and a quad of 220-ohm resistors (and some 1K-ohm resistors). One thing I've learned during this process is to verify everything before busting out the permanent-connection hardware; before soldering or gluing anything, I bread-boarded the LEDs and used a slight modification on the Arduino "Fading" demo to confirm that all four LEDs would light. This turned out to be wise; my initial plan had been to use a 1K-ohm resistor between all four cathodes (in parallel) and ground, but I discovered that allowed too much back-current to leak across the LEDs---not desirable at all.

Final configuration for the LEDs and resistors on the testing breadboard.

Once I'd verified the configuration, it was time to get to work. Trimming the anode leads a bit on the LEDs, I soldered one resistor to each anode. I then grabbed my Dewalt and 3/16" bit again and bored four parallel holes in the upper half of the case, then glued the LEDs in place. Due to a slight mix-up on my part, the green LED ended up with its resistor on the cathode, but no harm seemed to be done. I soldered all the cathode leads (including the resistor lead from the green LED) into one, and left the anode leads to be connected to the Arduino.

I ran into a bit of a conundrum at this point: with the LEDs living on the top shell, I wanted some flexible wire with some nice, solid end plugs to connect the LEDs to the Arduino control pins, but nothing in my kit really fit the description. Fortunately, I discovered that Radio Shack sells a SideKick basic kit that includes what the kit calls "breadboard jumper wire." It turned out to be the perfect tool for the job if I cut them in half and soldered the loose wire to my LED leads.

With the jumper wires in place, I connected the LED anodes to Arduino pins 6, 9, 10, and 11, and the cathodes to one of the two GND pins. I then wrote a couple of short LED test functions:

// LED control library, by Mark T. Tomczak
// Public domain
// To use:
// * Call SetupLedControl with an array of LED inputs IDs
//   (-1 terminates the array as a sentinel value)
// * Call functions in the library to control the LEDs.

// Control LED firing patterns
// -1 used as sentinel value
int *g_led_control_leds;
int g_led_control_num_leds = 0;

// *leds is the arraay of LED pin IDs, terminated with a sentinel value of -1.
void SetupLedControl(int *leds) {
  g_led_control_leds = leds;
  g_led_control_num_leds = 0;
  while(*leds != -1) {
    g_led_control_num_leds++;
    pinMode(*leds, OUTPUT);
    leds++;     
  }
}

// Blink several times (off-on-off).
// - times: number of times to blink
// - flash_delay_millis: Number of milliseconds to wait
//   before toggling the light on or off. void FlashLights(int times, int flash_delay_millis) {
  for(int i=0; i < times; i++) {
    for(int i=0; i < g_led_control_num_leds; i++) {
      analogWrite(g_led_control_leds[i], 255);
    }
    delay(flash_delay_millis);
    for(int i=0; i < g_led_control_num_leds; i++) {
      analogWrite(g_led_control_leds[i], 0);
    }
    delay(flash_delay_millis);   
  }
}

// This is a fun little function that delays a specified
// number of milliseconds, while showing the time remaining as the
// intensity of several LEDs.
// 
// When all the time is remaining, the LEDs all glow at maximum brightness.
// Elapsing time is shown as the LEDs slowly going out one-by-one. void CountDown(long time_to_countdown_millis) {
  unsigned long start_time = millis();
  // TODO(mtomczak): Arduino millis() function rolls over to 0 after 
  // about fifty days. You should account for this rollover... 
  // possibly build a timer library with some structs to hold that state.

  for(
    long time_elapsed = 0;
    time_elapsed < time_to_countdown_millis;
    time_elapsed = (long)(millis() - start_time)) {
    float percent_remaining = (float)(time_to_countdown_millis - time_elapsed) / (float)(time_to_countdown_millis);
    float led_fraction = 1.0 / (float)(g_led_control_num_leds);
    for (int i=0; i < g_led_control_num_leds; i++) {
      // The logic is exciting here, but it basically boils down to:
      // * consider the time remaining to be divided into a series of "ranges"
      //   (one per LED).
      // * If the time remaining is more than my range, glow maximum intensity.
      // * If time remaining is less than my range, shut off completely.
      // * Otherwise, glow proportional to how much time is left within my
      //   range.
      int output = 255;
      if (percent_remaining < led_fraction * (float)(i + 1)) {
        if (percent_remaining < led_fraction * (float)(i)) {
          output = 0;
        } else { // percent remaining is in range led_fraction * (i, i+1)
          output = (int)((percent_remaining - (led_fraction * (float)(i))) * 255.0 / led_fraction);
        }
      }   
      
      analogWrite(g_led_control_leds[i], output);
    }    
  }
}

I combined this library with the recognition code from my Arduino IR bridge project so that it would blink twice and then count-down over five seconds when it recognized an IR pattern. Overall, I'm not unhappy with the results!

What I learned

• This is the part of the project where the insulating properties of hot glue were especially helpful. I basically glued over the forest of wired-together cathodes so that they couldn't short against anything, and I'm pretty pleased with the results.
• Pay more attention to output ratings on LEDs. There's a noticeable visual difference between the green LED and the much brighter red LEDs (I'm guessing that my eye is reacting logarithmically to the difference between 6.3mcd and 10mcd). If I wanted to back this part out and try again, I'd go with four red LEDs.

Part three will wrap things up by adding the IR emitter and the code to drive the entire project.

Return of the Arduino part 1: mounting the controller and IR receiver

I've had an Arduino UNO board kicking around my house for about a year. My original plan was to create an input switcher for the TV that would run off the cable remote (it would maintain a bit of internal state and fire the commands to switch TV modes from the current one to the desired one instead of needing to fish around for the TV remote). This was pretty successful, and I published both the IR decoder and the input changer itself on GitHub. But I never got around to enclosing the project and making it "clean."

So this week, I finally purchased an enclosure and power supply for my UNO board. Trying it out, I immediately discovered that the enclosure doesn't really fit my Ethernet shield (which the reviews will tell you gladly; that's what I get for not reading the reviews before purchasing). Since I didn't need the shield, I shed it (I'll probably get another UNO to plug it into later).

Then, I got out my kit and got to work. It's been awhile since I had occasion to solder anything, and I've never done hot glue for an electronics project before. Overall, I'm not unhappy with the results.

I made a hole in the enclosure using a 3/16" bit on my Dewalt drill, angling it diagonally back-and-forth to get a good wide slot (I wanted to make sure the receiver had a nice, wide field-of-view to catch IR signals). I then seated the Arduino board in the enclosure and glued the IR sensor in place with my wife's glue gun (note to self: I've stolen my wife's glue gun and owe her a new one). My gluing technique is hellishly wasteful, but I figure I'll get better with practice. One thing I love about working with glue is that it doubles as an insulator; no more wiggling my fingers in between tiny wire runs to get the electrical tape on there!

IR decoder visible through the drilled slot in the side of the enclosure.

Once the glue solidified a bit, I did a quick soldering job and was all set. I wrapped the signal and power wires around the posts in the enclosure also and threw some hot glue on there for mechanical stability (I want to make sure any jostles the wires might take are being supported by the posts and not the Arduino's female pin coupler or the IR sensor's leads).

I then fished a sticker out as a finishing touch. :)

I'm very happy with both this sensor and its range. I ran some quick tests with the full enclosure at different ranges using my old remote signal decoder program, and found it was able to sense a good distance away (about 15 feet; plenty for my needs).

What I learned

• This IR sensor is more temperature-resistant than I'd feared it might be; I ran the soldering without any heat sinks, and no harm was done.
• Hot glue is actually pretty safe to work with; I'd worried I would get burned, but discovered that it appears to shunt its heat pretty quickly as it transitions to a solid state. solid strands can be touched within a few seconds of being emitted from the nozzle (though blobs should be given about thirty seconds to a minute to gel cleanly; they change opacity a bit on solidification, which is convenient).
• Hot glue is messy; spiderweb-strands everywhere, needed to go in with pliers to clean it up. Also, I'll be more careful in the future about letting the glue find its way into the seam of the enclosure itself; my enclosure no longer seats quite correctly because one blob of glue bubbled over the top and got into the joint.

Part 2 will discuss extending the software and adding status indicators.