"Screenshots taken from a version you do not own"

The Pixie-Net is a multi-channel radiation detector readout electronics module. Designed for high-precision coincidence gamma-ray spectroscopy using HPGe detectors, scintillators, or silicon detectors, the Pixie-Net offers not only detector signal digitization and waveform capture but also pulse height measurements, on-board energy histograms, time stamping and constant fraction timing, pileup inspection, external gating and online pulse shape analysis. Besides nuclear spectroscopy, the Pixie-Net can be used for neutron/gamma discrimination, time-of-flight measurements, and coincidence/anti-coincidence measurements.

The Pixie-Net is based on the Avnet MicroZed SOM, using a Xilinx Zynq as the key processor, and running a variant of Ubuntu Linux. Those are fairly popular devices and software, and there are lots of blogs and forums providing ideas, tricks, and tools for improvement. While the Pixie-Net comes with a working set of software for data acquisition, we continue to explore improvements. Some of them are listed below.

Disclaimer: As the subtitle of this blog is already a quote [1], so also the ideas below are largely based on what other people wrote and did before. Credit is given as far as possible, but no guarantee is given that the code modifications work reliably. These new functions are not (all/yet) part of the Pixie-Net firmware/software, but can be added in future versions and/or implemented by interested users. 

[1] T. Pratchett, "Only you can save mankind"; disclaimer on a fictitious video game

Javascript 2D histograms

January 2017

For the pulse shape analysis (PSA) variant of the Pixie-Net, results should be displayed as 2D histograms on a web page, plotting the detected pulses' PSA value vs Energy colored by intensity.  The PSA computes two sums over characteristic regions of the detector pulse, and calculates the ratio (PSA value) of the two sums[2]. The PSA value is then a condensed characteristic of the pulse shape, and can be used with suitable detectors to distinguish events or particle types, for example gammas and neutrons or gammas and alphas. A typical result, as the above measurement with an alpha/neutron source and a crystal stilbene detector, has distinguishable branches for different particle types. Neutron/gamma measurements with CLYC or liquid scintillators show similar results[3].

To plot the data, the free plotly javascript library was used to plot .csv data generated by the Pixie-Net. Note than in javascript terminology, "histogram" means accumulating individual data points into a histogram, whereas the Pixie-Net already produces histogrammed data that is plotted in what's called a surface plot. To see the source code of the plot and see zoom, export, and recalculation features in action, download the Pixie-Net software and open "psahistprojpage.html" in your browser. Firefox is recommended.

[2] NIM A 809 (2016), 132: http://dx.doi.org/10.1016/j.nima.2015.08.023

[3] http://www.xia.com/AppNotes/P500_PSA_firmware.pdf

 

 

PMOD GPS

UART communication

February 2017

The MicroZed has a "PMOD compatible" I/O connector, which is brought out as external I/O in the Pixie-Net. With some reconfiguration of the firmware and device tree [4], it is possibly to link the PMOD pins to the Zynq ARM's UART controller. On can then connect a UART GPS module to the PMOD port, here the Digilent PMOD GPS [5]--  though unfortunately, due to pinout mismatch, it can not be plugged in directly but needs cabling to swap TX and RX pins. By default, the PMOD GPS module operates at a baud rate of 9600 baud, with 8 data bits, no parity, and with a single stop bit. Luckily, this is the default setting for the UART0 serial port. So reading the GPS module's output is as simple as reading from the corresponding /dev via the command line.

Use dmesg | grep tty to check which serial ports are configured.

ttyPS0 is the USB/UART console, so use cat /dev/ttyPS1 to print out the GPS messages.

In my tests, the GPS found the time ok in the office, but for position lock I had to move things outside. The first "GPGGA" message indicates that the time is 16:57:59.090 UTC with "0" for position fix and "0" for the number of satellites used. The second "GPGGA" message indicates that the time is 19:23:22.000 UTC, with position 37.36... degrees N and 122.03... degrees W, "1" for position fix and "4" for the number of satellites used.

 

PPS reference signal

March 2017

When locked, the PMOD GPS module issues a Pulse Per Second (PPS) signal with a rise time of ~10ns, which can be fed into the Pixie-Net for external time stamping of the detector signals. Precision relative to the internal clock is about 1.5 microseconds. This is measured as the deviation from the average increment -- nominally 125,000,000 of the Pixie-Net 125 MHz clock ticks between consecutive PPS time stamps, but in reality 125,000,000 +/- ~250. The ~250 clock ticks jitter corresponds to deviation of up to ~2000ns with a more or less Gaussian distribution with 1.585 microsecond full width at half maximum (FWHM). 

 

The question is of course, is the PPS signal jittering, or is it the Pixie-Net’s clock. Checking the PPS signals from 2 GPS modules powered by the same PN on an oscilloscope, the PPS rising edges are in sync (always see 2 edges together on the oscilloscope) but the phase shift ranged between ~20ns and 200ns between scope captures in the ~ 10min observed, maybe shifting 10-20ns between subsequent captures.

[4] http://forum.xillybus.com/viewtopic.php?f=4&t=474&sid=cd7df1f2f54745f744cdf8834927684e

[5] http://store.digilentinc.com/pmod-gps-gps-receiver/

 

Upgrade to Ubuntu 15

March 2017

The Pixie-Net (at this point), is shipped with Xillinux, a version of Ubuntu 12. This is a nice, easy way to get started with the Zynq/MicroZed and free to use with the Xillybus Lite kit. (After all, we are using the Zynq because we want to integrate FPGA and ARM/Linux, and almost everybody else makes it ridiculously hard to share data between them, if they mention it at all.) But for some of the more advanced functions below, it's useful to upgrade to something newer. This involves such fun [man made] problems as rebuilding the kernel, cross compiling, remaking device trees and so on.

I list here and here some references I found useful for that. The procedure is quite cumbersome to set up the first time, but mostly following recipes. Contact XIA support if you want to try a copy of the Pixie-Net Ubuntu 15 OS variant.

 

 

Login via Remote Desktop

April 2017

Normally, login to the Pixie-Net's Linux system is via a terminal, either via USB/UART or via SSH. There are Zynq modules that have video outputs, but not MicroZed used here, and also use of the video may require licenses for video drivers. However, as pointed out here, it is perfectly possible to run a graphical desktop internally on the Zynq, and log on to it via Remote Desktop Connection. So what if the board has no connector for video output -- just view it over the network. (Note: The Pixie-Net's micro HDMI connector is used for GPIO, not video)

So instead of using this ....

... you can use this. (Note the same IP number on the window frame.)

Feel free to read your emails, write reports, and play games right on your Pixie-Net. You can even open a terminal window and start typing the same things you would type into the SSH terminal ;-).
I here chose xfce as the graphical desktop, but xrdp or Ubuntu Desktop would probably work just the same. Probably requires something newer than the Xillinux / Ubuntu 12 shipped by default with the Pixie-Net -- I used Ubuntu 15, see above.

Here is the procedure:

In the Linux terminal on the Pixie-Net, install as follows (Ubuntu Desktop still provides some basic apps):

  • sudo apt-get install --no-install-recommends ubuntu-desktop

  • sudo apt-get install xfce4

  • echo xfce4-session >~/.xsession

In the Linux terminal, after each boot, find the Pixie-Net's IP address:

  • ifconfig

On a remote Windows PC:

  • Run Remote Desktop Connection.
  • Specify the Pixie-Net’s IP address
  • Log in

On a remote Linux PC:

  • use X11 forwarding by typing ssh -Y root@192.168.1.58 (or equivalent) in a terminal

 

 

Using ROOT on Pixie-Net

April 2017

With the graphical desktop, one of the programs one can now run is ROOT. It's available as an Ubuntu package. Just install

  • apt-get install root-system-bin

If you really want, you can view (and even analyze) the Pixie-Net data right on its own processor (screenshots coming soon).

Do I recommend it?   Hmmm, not really. But it can be done. And maybe someone will someday make a graphical user interface for the Pixie-Net data acquisition that way ... to click buttons instead of typing ./startdaq.

Update August 2017

A ROOT demo interface now has been developed. Still very basic, it can configure the Pixie-Net (./progfippi etc), view ADC traces, start data acquisitions (./startdaq), and display the resulting spectra. The code is contained in a ROOT file pixieNet.C (available on request). To use it, simply start ROOT (in a terminal, type root) and then start the GUI (type .x pixieNet.C).

 

 

 

IEEE 1588 Precision Timing with a Pixie-Net

PTP with Zynq Hardware Timestamping

May/June 2017

The Zynq used on the MicroZed used on the Pixie-Net comes with some basic built in IEEE 1588 Precision Timing Protocol (PTP) functions. That can be used to synchronize the data acquisition between multiple Pixie-Net units to nanosecond precision. (Depending on the type of implementation, that's hundreds or tens of nanoseconds.)

While the Zynq provides HW timestamping functions for PTP, the clock adjustments have to be controlled by software. Open source programs for that purpose include ptpd and LinuxPTP. While ptpd is an Ubuntu package (even in Ubuntu 12), LinuxPTP has to be downloaded and installed (at least in Ubuntu 15). But it seems to have better hardware timestamping support (at time of writing) so the tests below were done with LinuxPTP.

For Ubuntu 15, the following kernel options were required for PTP with LinuxPTP:

  • CONFIG_PPS
  • PTP_1588_CLOCK
  • CONFIG_NETWORK_PHY_TIMESTAMPING
  • CONFIG_XILINX_PS_EMAC
  • CONFIG_XILINX_PS_EMAC_HWTSTAMP

and the (older) xemacps network driver has to be specified in the device tree. This is the one used used in Xilinx' PTP "tech tip" example. (With the new Cadence driver, PTP is not supported for the MicroZed's Zynq, only for the UltraScale.)
After installing LinuxPTP, it can be run with 2 Pixie-Net units connected through the network :

and the software reports time offsets between master and slave of on average ~17ns, with a FWHM distribution of ~1300ns:

Not bad for something that is included in HW and SW for free, and with some stretch, "1300ns" is already "hundreds of nanoseconds" (13 of them). But the performance is not really yet up to the level of nuclear physics timing needs. And so far only the Linux clocks are synchronized, not the clocks used by ADC and FPGA to capture and process detector signals.

But improvements are in progress ...

PTP with Ethernet PHY Timestamping

October 2017

Since PTP timestamping is best performed as close to the hardware as possible, a new hardware revision of the Pixie-Net has an optional secondary Ethernet PHY that can generate clocks and triggers from the PTP clock. The new Pixie-Net still uses the MicroZed with its default PHY, so the Zynq's MII interface was rerouted through the FPGA fabrics to the new PHY similar to the description here. The PHY is supported by LinuxPTP and the registers controlling clock and trigger outputs can be accessed with mii-tool. It can even turn on synchronous Ethernet. Using the PHY output clock for the ADC and the FPGA processing logic, detector pulses can be captured synchronous to the rest of the network. Precision reaches between a few hundred nanoseconds and a few hundred picoseconds depending on network capabilities and signal source.

The plot below shows the time difference measured for 2 coincident gamma rays captured in two Pixie-Net modules synchronized with PTP. One can see the time difference drift due to small clock frequency differences, then reverse due to the periodic adjustments by LinuxPTP.

This poster ( NSS_NTSposter.pdf ) summarizes the performance measurements in a number of configuration, including a comparison to clocking the Pixie-Net modules with a network clock from a White Rabbit demo system.

 

 

 

 

MATLAB Interface

August 2017

A few days experimenting with MATLAB resulted in the demo interfaces for a host PC, as shown above. It implements

  1. Operate Pixie-Net via serial port (click button in MALAB rather than typing command in terminal)
  2. Read resulting data files
  3. Display data from files.

While in principle the Pixie-Net's Zynq firmware can be developed with MATLAB tools as well, this is currently not supported by XIA.

 

 

 

Webpage Operations

August 2017

 

As an alternative to the terminal entry and execution of the C programs to set up and perform data acquisition (e.g. ./progfippi and ./startdaq), there is a [moderately] secure web page that allows execution of the C programs as a cgi script from a webpage. The Web Operation page can be reached via a link in the “CGI Current Data” section of the Pixie-Net Home page. It essentially lists the C functions that would normally be typed into the terminal; clicking on the function name executes the function. Data is created in a separate subdirectory (/var/www/webops) with Linux permissions for data write access for the webserver. The Web Operation page shows the equivalent links from the Pixie-Net home page to download and display the data.

As this webpage allows generation and execution of files from any remote user, the webserver should also be modified to require authentication to access this webpage. See example here.

 

 

Spectrometer Performance

September 2017

Below are a few plots showing how the Pixie-Net compares to the other Pixie spectrometers in some typical nuclear physics setups

Energy Resolution with HPGe Detector

Pulse Processing Throughput

 

Timing Resolution with Scintillators and Pulsers