Firmware Release v0.33

About a week ago, the latest version of the firmware was released. An overview of the main changes can be found in the release notes here. Included in the release was support for the branch-on-trigger and branch-wait features discussed in John’s post. Additionally, some of the DHCP work done by Jeff was included.

icnt instruction

A major feature of the new firmware release is that it includes a working implementation of the icnt instruction, which can be read about in David’s post. The basics of the implementation were already in place, but a few tweaks were necessary to get the features fully working. Specifically, the following changes were made:

The signal for branching based on a comparison between a counter register and another value had some delay issues, which were fixed.

The signal for writing data for the icnt write instruction was unnecessarily delayed, which was fixed.

The counter registers were experiencing some non-deterministic behavior due to being affected by two clocks (the board clock and the input triggers), causing them to sometimes reset erroneously. This was fixed by just using the board clock and watching for edges in the input trigger.

The data output from the processor wasn’t hooked up to the SRAM controller, so data from the icnt write instruction never made it to the SRAM. These signals were hooked up, enabling the processor to write to SRAM.

Tests

A few tests were run to make sure the instructions worked correctly, including a test that counted the number of times a button was pushed, and a test that used a feedback loop that fed output signals from the board into the input triggers, which let us test that the inputs were counted correctly and were written to memory correctly.

All of the tests worked when run on the board in the CSE lab. However, it currently looks like there are some timing issues on a different board, which causes this firmware (and v0.32) to not work. This problem is being investigated.

Networking the Sequencer Board

Overview

The pulse sequencer board communicates with computers over a standard 100 Mbps Ethernet interface using the Pulse Transfer Protocol (PTP).  The original firmware determines the IP address from the device ID (set by DIP switches 2-5) when DIP switch 6 is flipped down.  This allows one to easily determine the IP address of a sequencer board, but is problematic if another system on the network is using the assigned IP address.  When DIP switch 6 is flipped up, the sequencer board will find an unused address, avoiding this problem, but now one has to send a broadcast packet from the computer to find out what the address of the board is.

The usefulness of this feature is highly debatable, since it addresses problems for which there are already effective workarounds.  Most users have a router for each sequencer board, meaning that if the board chooses its own IP address, it’s not likely to choose an address that’s already in use.  Implementing these features will hopefully make the firmware more robust, however.  This operation is thus more of a “keep Jeff busy” task until the physics community gives me something more urgent to do.

API Change to Support Multiple Devices

This change applies to the Python software in the “pulse-sequencer” SVN repository.

In the past, the networking API for the test configuration (/pulse-sequencer/software/python/test-config.py) did not support addressing multiple devices.  Device discovery was already implemented in the discover_devices() method defined in /pulse-sequencer/software/python/sequencer/ptp/__init__.py .

I made a change to this API to support multiple devices.  Now, when you call setup(), it calls discover_devices() to determine which device IDs correspond to which IP addresses.  You can call select_sequencer(device_ID) to “select” that device, meaning all subsequent commands will be sent to that device.  To choose a different device, simply call select_sequencer() with the device ID of the device you want to use next.

This change has (probably) not been committed yet, because discover_devices() is a time-consuming process.  It’s less than five seconds, but runs every time setup() is called.  I’m not sure if physicists’ scripts are calling setup() only once per experiment (as they should), or if they’re calling setup() thousands of times — which could incur a huge delay.  I’m planning to introduce a workaround where setup() caches the discover_devices() data, and simply re-uses that data whenever it’s called again.  If it detects the data is invalid, it will call discover_devices() to set things right.

Dynex Router Support

The sequencer board is designed to operate on a network.  In order to do so, it needs to obtain an Internet Protocol (IP) address, which identifies it on the network.  To obtain an IP address, it uses the Dynamic Host Configuration Protocol (DHCP) to communicate with a DHCP server.  The server offers an IP address that isn’t currently being used by something else, and the board accepts and stores it.  This process consists of sending four messages:

1. DHCPDISCOVER – the sequencer board sends a broadcast message asking for an IP address.
2. DHCPOFFER – the DHCP servers on the network offer IP addresses to the sequencer board.
3. DHCPREQUEST – the sequencer board contacts a specific DHCP server to confirm its offer.
4. DHCPACK – the server ACKnowledges that the sequencer board has accepted the address.

The old sequencer board firmware only worked with Linksys routers.  When we tried to use a Dynex router, the sequencer board could not obtain a DHCP lease.  The problem was a malformed DHCPREQUEST message: the board assigned itself the offered IP address BEFORE sending the DHCPACK message.  The Dynex router didn’t like this, so it kept sending DHCPNACK  messages back — to indicate failure.

The fix was remarkably simple: I added DHCP Option 53 to the DHCPREQUEST message, to indicate the IP address the board chose.  I also wrote a hack to force the DHCPREQUEST messages to come from 0.0.0.0 instead of the board’s chosen IP address.  It worked almost immediately, and I completed this task within a single lab period — a grand surprise given that hardware labs in class often take several lab periods.

DHCP Lease Renewal

When a network device like the sequencer board obtains an IP address from a DHCP server, it doesn’t get to keep the address forever.  The DHCP server sends a lease time, in seconds, indicating how long the sequencer board gets to have its IP address.  At the end of this lease time, the DHCP server may give the address to another network device.  If this happens, two devices will have the same IP address, which is never supposed to happen!

To resolve this problem, I’m working on implementing lease renewal.  When the lease time is almost over, the board will need to ask the DHCP server to extend the lease time, preventing the server from giving the board’s IP address to another system.  This system involves adding some logic to the DHCP module, and adding a counter to track the time.

Logic elements are the small pieces on the FPGA that make up the circuits in the firmware.  The Altera QuartusII compiler configures these logic elements to make the FPGA behave in the way we specify in VHDL code in the sequencer-vhdl repository.  We’re using an Altera Cyclone FPGA, which has a measly ~12,000 logic elements.  I say “measly” because we’re at 87% utilization — we have around 1500 logic elements remaining.  Since we would like to add new features to the chip, we need to make the most of our remaining logic elements.

The lease counter needs to hold a maximum lease of about 1 day.  If the server sends a longer lease, it gets truncated.  1 day = 86400 seconds, which requires 17 counting bits.  Since our board clock operates at 100 MHz, we need to count 100 million clock pulses for each second, so our counter will require 43 bits.  Even with optimizations, that’s still a big counter — it required about 120 logic elements when I coded it in Verilog.

To make the counter smaller, I implemented a Linear Feedback Shift Register (LFSR).  Like a counter, an LFSR changes its combination of bits on each clock cycle.  If you have 2^n memory bits, a LFSR will cycle through (2^n – 1) different combinations of bits, using only n registers and one additional logic element.  However, there’s no straightforward way to determine how many clock ticks have occurred by examining the LFSR’s current state, unless you use more logic elements than a regular counter would require.  If you know an LFSR’s start state, you can calculate its state after X clock ticks while you’re designing the hardware, and use the counter to tell you when X clock ticks have passed.  This means I could use an LFSR to tell me when a second has passed, and use a regular counter to count the seconds remaining in the lease time.  This implementation required ~96 logic elements when I coded it in Verilog.

Couldn’t I track the time in minutes or hours instead of seconds?  That could eliminate some bits, but remember: the lease time is transmitted in seconds, so I would need circuitry to convert it.  Also, why bother with such a complicated design when a mere 20 logic elements are at stake?  I thought it would be an interesting exercise to see if this optimization would really make a difference.

Future Projects

Connecting and Disconnecting from a Network

Currently, the sequencer board doesn’t appear to detect when it has been connected and disconnected from a network.  This detection is important because the network module needs to “reset” itself when the network goes down.  Otherwise, it might try to renew its IP address on a network it’s not connected to, or assume that it still has its old IP address if the router it’s connected to gets rebooted.

The obvious workaround is to power-cycle the sequencer board (turn it off, then on) after plugging in a network cable or rebooting a router.  However, this could be tedious if you have lots of sequencer boards.  Given that most people don’t change their networking configuration very often, though, this might not be a significant issue.

Remote Access

Ah, the dream of every physicist: running experiments while sitting on the beach!  Remote access to sequencer boards would allow physicists to operate them without being in lab, which could reduce transportation costs and allow more people to share a trapped ion quantum computer setup to run experiments.

The complexity of this setup depends on the connection between your computer and your sequencer board.  If you are connected via a shared public network, for example, you need security.  You can purchase a Linksys VPN router here for just $99 ($50-ish used) that lets you connect to that router from anywhere (using a username and password).  You’ll need to make sure it works with your organization’s Internet setup (some organizations have firewalls that don’t allow you to connect from outside their network).  You could also try to download new firmware onto your existing router, but that runs the risk of “bricking”, or ruining, your router.

We could try to make the pulse transfer protocol routable, but we would still have to encrypt it somehow, and the encryption hardware would (probably) not fit on the FPGA.  If you’re interested in remote access, let us know!

leading up to squint

Posted by Paul Pham

In a few days I’ll be leaving for SQuInt, just a man, a van, a plan, and Panama. Don’t be surprised if there is a renegade poster or private demonstration of a yellow box, or if people are waving around FPGA sequencer boards. For the past few weeks, Rob, David, and I have been hard at work getting the input counting feature to work in time for its big unveiling in Santa Fe. I am fully confident that we’ll finish testing it before we leave.

Software support is already added in the v1 Python compiler, and the latest version in Git works for four out of five subopcodes (counter reset, counter latch, counter compare to a threshold, counter branch-on-compare). The last subopcode, counter write-to-memory, is tricky because it involves stopping the PCP fetching instructions from memory, writing to a new address (via an address accumulator that increments by one each time), and then resuming the PCP instruction pipeline at the same address as before. As you would expect, there are issues with delay slots (just insert more nops!), converting addresses between 32-bit (use by PCP) and 8-bit (used by PTP), and clocking issues (synchronizing between triggers and the system clock).

The other members of our team have also been busy in the meantime.

Jeff is finishing up new support for network device discovery. No more hard-coded IP addresses or dedicated routers! With his new changes, the sequencer can get a dynamic DHCP address from any gateway / router, count down the lease time, and then release and renew its IP address. This, combined with re-enabling the ICMP (ping) module, will make the sequencer a first-class network citizen again. He has also added support to the Python v1 compiler for discovering all sequencer devices on the network dynamically, and selecting between them via the software API when loading pulse programs. I won’t reveal any more, but hopefully while I’m gone he’ll write up his own blog post.

David and John have started work on the clock board, which should make the existing system a little more compact and less like a cable jungle. This is their very first circuit design, using one of my favorite open source projects, gEDA/PCB, complete with 70s X11 widgets and arcane keyboard shortcuts. Some day, when you are a crotchety old man, you can also make your students use weird toolchains because it is cheaper and builds character.

Peter worked on designing a proposal for pcp5 to support parameter scanning, which is sleeping for now while physicists decide if they really need it. But arithmetic and register support are pretty useful, and I am frustrated that ginormous hard-coded sequences are necessary for parameter scans. Eventually, my engineer’s sensibilities will boil over and force the issue. For now, Peter is working on modifying the firmware build system to generate multiple binary targets. This is the first step to creating alternate configurations and even unit tests for firmware.

Finally, our own Aaron Avril will be presenting a SQuInt poster on his yellow box. Here are some candid photos in lab, courtesy of Jeff Booth. See if you can guess what all the parts are.

Wait-Trigger & Branch Trigger Level Detection

Posted by John Williams

This post covers the addition of two requested features to the Python v1 Compiler and the underlying pcp32 firmware.

• Branch-on-trigger Level Detection
  • Python API Call: branch(label, triggers, level = 1)
  • PCP32 Instruction: btr [triggers] [level] [address]

• Branch Wait
  • Python API Call: branch_wait(label, triggers, level = 1)
    

Branch-on-trigger Level Detection

One feature that has been requested is the ability to select the level to trigger when using a branch event.

When this flag is set to zero, the trigger mask will select bits of the trigger inputs that must all be low in order for a trigger to occur. If all of the bits of the trigger mask are zero, the default behavior is to never trigger regardless of whether high or low level detection is used.

Branch Wait

The branch-on-trigger (btr) instruction was previously used mainly in loops to spin the processor while waiting for some trigger input.

It was discovered that this functionality could be duplicated by taking advantage of a property of the halt instruction that causes the last instruction in the pipeline to be executed every cycle.

The branch_wait event will be compiled down to the following sequence:

halt, nop, nop, nop, nop, nop, btr

filling all 6 branch delay slots and achieving the desired behavior.

Note: These features are not included in the recent 0.32 release but are in the repositories and will be included in all subsequent releases; additionally, the new level argument is optional for each and will default to the previously assumed value of  ’1′

Input Counting Instruction added to the Python Compiler v1

Posted by David Nufer

Overview

The Input Counting instruction (icnt) has now been added into the Python Compiler v1 and is available on the SVN.  The icnt instruction has 5 subopcodes that define different functions:

• Reset — resets one of the input counters.
• Latch — latches the current value of one of the input counters to its associated register.
• Write — writes the value of one of the input counter registers to the memory address in the address accumulator, and increments the address accumulator.
• Compare — compares an input counter register to its associated compare register and sets the compare flag it the comparison is true.
• Branch — branches if the compare flag is set.

New API Calls

There are five new API calls for the icnt instruction, one for each function:

• reset_input_counter(input_channel) — input_channel :  input trigger
• latch_input_counter(input_channel) — input_channel :  input trigger
• write_input_counter(input_channel) — input_channel :  input trigger
• compare_input_counter(input_channel) — input_channel :  input trigger
• branch_input(label) — label : the instruction to jump to

There are 8 input triggers are defined in /software/python/sequencer/constants.py, one for each input channel.

Test Script

There is a small test script called test_input_counter.py that uses all the new API calls.  At first glance, it appears that all the new API calls compile properly.  The test script is also being used to test that the firmware implementation of the icnt instruction is working properly.

A Few Firmware Fixes

Posted by Rob McClure

Arbiter Fix

As mentioned in a few previous posts, the latest firmware would compile but would not run correctly. The board would shoot off 3-5 malformed ARP packets and the boot LEDs would not cycle through their display. Various things were considered as possible problems: perhaps the power-on-reset wasn’t working, perhaps the switch to a new compiler (Quartus 9.1 from Quartus 5.0) messed something up, or the asynchronous FIFOs were malfunctioning. Turns out that none of these were the problem (although the power-on-reset actually wasn’t working. More on that later in the post). The problem was actually the arbiter, which lets two or more masters control the same slave. The arbiter has a timeout so that it isn’t stuck waiting on a master that doesn’t actually have data. For easier debugging purposes, the timeout was set to 10 clock cycles, which in the time frame of the things the arbiter controls is really, really fast. The timeout in the previous firmware was 500,000 cycles, and reverting the timeout to that value solved the problem. The network controller had a couple of arbiters in it, and the I2C controller also had a few arbiters (and other places that probably had errors we didn’t see). Such a simple problem caused so many pieces to malfunction.

Power-on-reset Fix

As was mentioned, the power-on-reset still didn’t work. So, for the firmware to work, it had to be manually reset (with the switches on the board). While not a pain, it would still be more convenient for the board to reset itself. As shown here, the reset is just two T flip-flops linked together. By using SignalTap to examine the signals in the module, we gathered the following data:

What is shows is that the second flip-flop starts up in the finished state, instead of doing the one toggle it is supposed to to generate the reset pulse. It is possible that the compiler does some optimization that eliminates the important time-component of the flip-flop function. By inserting a third flip-flop into the module, the reset pulse was generated correctly, as shown by the updated SignalTap data:

As a result, the board now boots up and resets itself correctly, so it functions without any extra work by the user.

sequencer v0.32

These fixes mean that a new version of the firmware has been released. v0.32 is available from the Sourceforge page. Currently, it does not include the power-on-reset fix, but that may change in the future. The version with the fix can be found in the Git repository.

Linux hexdump behavior

Posted by Peter Sullivan

There were reports that the software was not working on Python 2.6 with Linux.  I confirmed that the data being sent to the device is correct and that there should not be any issue with running the software under Linux.  The –save option will store the correct binary representation of what is sent over the network as well, however, it should be noted that at least under Ubuntu the default setting of hexdump is 2-byte hex, running hexdump -C or hd instead should give the desired 1-byte hex representation.

Clock Programming

Posted by Jeff Booth

Overview

One of the components of the pulse sequencer is a clock source.  The source must generate an 800 MHz clock signal for a sine wave generator, and a 100 MHz signal for the processing unit.  Originally, this clock source was provided by an external signal generator, which was both expensive and bulky.  As an alternative, Paul bought a clock source evaluation board, which can be programmed via a serial interface to generate a 1600 MHz clock signal.  Unfortunately, this board must be programmed every time it is powered on, requiring a computer and a serial cable.  Jeff’s project is to program the clock from the processing unit when the system powers on, and then convert the 1600 MHz signal to 800 MHz and 100 MHz signals.

The Clock Source

For the clock source, Paul bought an ADF-4360-2 chip on an EB1 Rev. D3 evaluation board.  This board has a connector for a 9V battery; since the battery died, we connected it to a 12V power source (since the on-board voltage regulator can handle it).  It has a clock output labeled “RF” and a DB9 female connector for communications.  The pin mappings are:

3. CLK – Clock
5. Data
7. LE – Latch Enable
8. GND – Ground

Every time CLK transitions from low to high, the bit on Data is read into the least-significant bit of a shift register.  Once 20 data bits are read, followed by 2 address bits, LE should be raised high.  This will cause the data in the shift register to be transferred to the register indicated by the address bits.  There are three registers that must be programmed in this order: R, Control, then N.  Here are the 22-bit values to shift in for each register, which Jeff found by clicking the “Registers” button in the clock source’s control program on an attached desktop computer:

R: 0000 0000 0000 0000 1100 1001
Control: 1000 1111 1111 0001 0000 1100
N: 0000 0000 1111 1010 0000 0010

Clock Divider

The AD9513 clock divider is programmed by 11 pins, each having one of 4 different voltage levels.  Our desired configuration produces CMOS voltage levels on two ports.  Port OUT0, on the connector labeled J1, divides the clock input frequency by 16.  Port OUT1, on connector J3, divides the clock input frequency by 2

Jumper configuration on the clock divider board

DB-9 female connector from the computer; plugs into the clock source.

google610888f721ab7e8a

new year’s progress

Posted by Paul Pham

A brief post before bedtime, since we had a good night of pulse programming, and it has given me a high that I can normally only get by running.

New sequencer boards have finally arrived, let me know if you want one! They got here on Christmas Eve, here is a picture of them arranged decoratively in a wreath on my apartment floor. The tenth one arrived a few days later.

A wreath of sequencer boards

I’ve tested one so far and it works great. Screaming Circuits did a very professional job, and for cheap, I highly recommend them.

Also, school is now back in session here at UW for the winter quarter. Jeff Booth has taken a bunch of cool photos of our lab and the undergrads who labor on your behalf. I definitely approve of all the dignified poses, which befits the gravity and seriousness of our mission.

Rob McClure

Rob is debugging the firmware to work with the latest version of Quartus, the first new compile in over two years. This is a necessary step to testing our new icnt instruction, as well as keeping our firmware up-to-date and efficient.

John Williams

John is working on the branch-on-trigger instruction to enable masking certain inputs as well as triggering on both high and low logic levels. He discovered that we could achieve a wait-on-trigger event by calling halt and then placing a branch-on-trigger instruction in the delay slots afterward.

Jeff Booth

Jeff is working on the programmable 2.0 GHz clock source and the divider that produces an 800 MHz clock for DDS and a 100 MHz clock for the sequencer board. He made great progress just tonight configuring the clock source via its serial interface using the I2C lines on the breakout board. This is awesome because we can use a tiny Analog Devices eval board instead of a gigantic synthesizer that costs hundreds or even thousands of dollars. Moreover, we can configure it on power-up using a simple Python script, instead of requiring the clunky Windows control software and a parallel port. I’ll let Jeff brag about his achievement in his own blog post.

David Nufer

David is working on adding support for the new icnt instruction to the Python compiler, which is the most complicated instruction to date. It has five subopcodes, requires two new register files, and an address accumulator for writing data to memory. We won’t be able to test or even use the new icnt instruction in firmware without this software feature.

Peter Sullivan

Also, we are proud to introduce a new cast member, Peter Sullivan, who was an undergrad with me in Boston. Tired of the stuffy pulse programming scene on the East Coast, he is now out here to see how we modulate our frequencies on the Left Coast. He is currently working on some software compatibility problems between Python versions 2.5 and 2.6 across Windows and Linux.

You can see the other photos in this new Flickr set.

Happy New Year! As you can see from the first few days of 2010, we are on track for another year of ruthless, inexorable progress.

Just a song before we go on winter break

Posted by Paul Pham

Some of you probably don’t celebrate Xmas, but you probably eat cookies. And cookies, like pulse programming, have universal appeal, almost hardwired into the human psyche. The evidence? Behold, Exhibit A:

It is too bad that not all evidence is this delicious. I already ate the constant-amplitude sine wave since, let’s be honest, there was too much reflection noise in the icing to feed into the jello AOM or modulate other baked goods.

This coming week is finals week here at UW in Seattle, and then there is a two week fantasy where no teaching occurs until January 4th. It is a time to reflect on the past year’s research and to look forward to the next funding cycle. Good luck to all the undergrads in their exams, and to Aaron and me in our grad school applications. My safety school is the one with seven bells, who toured Seattle in May.

Okay, project updates!

SEQUENCER BOARDS

I mailed these off to Screaming Circuits last Monday (12/7) from Seattle, they arrived on Tue (12/8) in Canby, Oregon, and they should ship, completely assembled, at most ten business days later (Tuesday, 12/22). Then everyone who ordered a sequencer board, and was very good this year, will get one shortly after the Winter Solstice. Unless they are visiting from out of town, then they can just take it back with them on the plane.

PULSE PROGRAMMING ENGINEERS

In our ongoing saga of bringing engineers and physicists together, Dave Bacon (the Quantum Pontiff) generously gave an introductory lecture on quantum computing this past Friday afternoon (12/11) in the CSE building’s 4th floor breakout. I will be paraphrasing my notes from this talk in a separate entry, one where the WordPress latex plugin is installed. C’mon, WordPress, get it together! Like we are going to rely on MathML and download new fonts.

The CSE undergrads have made great progress in getting an Analog Devices 2.0 GHz clock source running with a divider down to 100 MHz, and also blinking LEDs on the sequencer board. We went over the source code for the sequencer firmware in VHDL on Thursday. Next up, the undergrads will be splitting up to work on individual projects after the winter break, such as:

• programming the clock source over I²C instead of using a PC and serial/parallel cable
• getting the sequencer firmware to compile into a working binary using Quartus 9.x
• adding support for the new input counting instruction to the Python compiler (v1)

I will also be e-mailing my individual contacts at all known ion trap groups to see if they have any interesting new projects coming up that they need engineering help with.

INPUT COUNTING

The new firmware which supports the icnt (input counting) instruction and the pcp4 machine compiles in Quartus 9.x, but the resulting binary produces garbled ARP frames when programmed into the sequencer. If I had to guess right now, I would say it is due to the asynchronous (double-clocked) FIFO that buffers in between the Ethernet 25 MHz transmit/receive clocks and the internal 50 MHz system clock that runs the FPGA network controller. Maybe this is compiled differently in Quartus 5.0, the compiler version used in the last known working build (v0.29).

Also, I finally broke down and pushed my input counting changes to the public git repository, in the branch called “icnt”. I know I threatened to charge all of you ruinous sums of money for this code, but I will just have to gouge you some other way. Also, it is a real pain in the ass to merge this into the master branch right now, so you have to pull the “icnt” branch specifically into a fresh local working directory, and who even knows if I’m using git branches correctly. I am horrible at keeping secrets, which is why I made this project open source in the first place. Also, I’m Iron Man (spoiler alert).

SUPERCONDUCTING QUBITS

Dave finally explained to me that superconducting qubits can all be different due to manufacturing tolerances, and the great thing about ions is that they are all the same. And then I remembered, didn’t we (meaning John Martinis and Steve Waltman and me) design the sequencer board to be general enough for them there Josephson junctions in the first place? That is why there are all those fiberoptic connectors on the board, to reduce noise. However, I still don’t really understand how Josephson junctions work, and they might need sub-nanosecond timing resolution. I think I said that in my thesis five years ago, so it must be true.

But the mystery (???) is that I know the sequencer board, with its 10ns resolution, is already being used in superconducting qubit research at UC Santa Barbara and possibly at NIST Boulder (!!!). Incidentally, nobody in those places uses our firmware or DDS-based frequency synthesizer, so some evangelizing about open source science will be needed. I will do some sleuthing to get to the bottom of this, and report back to you soon.

Until next time! –Paul