Text file format

The background data is recorded in a ASCII format, readable with any standard editor. Every background record contains some header information (like time stamp, ...) and up to five background variables for every pixel. The actual recorded variables depend on the parameter settings of the recorder.

Background variables:

Background level determined by the head electronics current monitor (key: "Sigma-Delta"),
Background level determined by a variance analysis of the PMT signal. (key: "HW Statistic"),
Background level given by the threshold of the first level trigger logic. (key: "Threshold"),
Pedestal of the PMT signal. (key: "Summ"),
Trigger rate of the first level trigger. (key: "Hit Rate")

Every background record originates from a single telescope and contains always one header line. This header gives the following information. The values are taken from the sample file shown below.

Sample time used for recording of the background information [s]. (here: 5s)
Time stamp of the background record. The time stamp is given by the second counter and the subsecond counter of the electronic at the beginning of the recording of each record. (here: 700448327s, 634 x 100us = 63.4ms)
Duration of the download of all data required for the background record. The duration is given in the same unit as the subsecond counter. (here: 10074 x 100us = 1.0074s, this is a typical value, if all varaibles are requested.)
The number of the telescope where the data comes from (here: Telescope 2). Note: The order of records coming from the telescopes in the file is not necessarily the same at all sample times.

Data format of the background variable entries. Every entry contains a header line with the key, some parameters and the information how many pixel are covered.

The entry "Sigma-Delta" contains the raw values of the current monitor. There are no values for virtual channels. Only data recorded with the prototype phase telescopes will contain this information.
The entry "HW Statistic" contains the variance the PMT signal noise in units of (ADC counts)^2 * 1000. The parameters offset [ADC counts] and the number of samples used to calculate the variance are also given. The time resolution of the hardware ADC signal is 100ns. (here: The first pixel has variance of 2.748 (ADC counts)^2, the offset was 450 ADC counts and 64000 samples = 6.4ms of data were used)
The entry "Threshold" gives the threshold of the first level trigger in units of (N * ADC counts), where N is given by the number of elements in the moving average filter (default value is 10). (here: The threshold of the first pixel is 504.8 ADC counts)
The entry "Summ" contains the pedestal calculated by the hardware statistic functions. The same parameters are use as given in the entry "Variance". The pedestal can be calculated by the formular: pedestal = sum / samples + offset. (here: The first pixel has a pedestal of 3068158 ADC counts / 64000 + 450 = 497.94 ADC counts; the pedestal of the prototype is shifted to about 500 ADC counts to be able to record possible undershots of the PMT signal).
The entry "Hit Rates" contain the trigger rate of the first level trigger logic in units of Hertz. Values around 100Hz should be normal for most runs. (here: In the beginning it is possible to have too high thresholds. In this case the trigger rate will be zero)

All background data is produced by the classes TsBackgroundBase, TsBackground and Ts4Background coming with the hardware access library fdhwlib. It contains the methods saveAMFormat() and loadAMFormat() to write and read the background data also in the ASCII format defined by A.Menshikov.

Sample background file

The following lines show a shortend sample background file. It contains the entries of one record and a little sample data. The listing is cut after the beginning of the second record.

** Timer is set = 5.000 s GPS time = 700448327 s Subsec = 634 x 100us Duration = 10074 x 100us TelescopeId = 2
** Sigma-Delta. Row values [20][22]
       0       0       0       0       0       0       0       0  ...
       0       0       0       0       0       0       0       0
...
** HW Statistic (Variance * 1000 )[20][24], offset = 450, samples = 64000
       2748       2698       2443       2503       2815       2943  ...
       2820       2509       2559       2464       2632       2765

...
** Thresholds (Row values) [20][22]
    5048    4912    4924    4998    4999    5124    5092    5141  ...
    4997    5011    4902    5095    5110    4995    5048    4455
...
** Summ over 64000 (Row values) [20][24]
  3068158  2199110  2317441  2772266  2611344  3526464  3286978   ...
  2690289  2829912  2418027  3340485  3481613  2595680  2942420
...
** Hit Ratess (Row values) [20][22]
       0       0      14       4       0       0       0       0  ...
       0       0      65       0       0       0       0       0
...


** Timer is set = 5.000 s GPS time = 700448332 s Subsec = 966 x 100us Duration = 10768 x 100us
** Sigma-Delta. Row values [20][22]
       0       0       0       0       0       0       0       0
       0       0       0       0       0       0       0       0
...

Background data in the Performance Monitoring Database

For the FD performance monitoring the background data is stored in a database. The organisation of the background data in the database is choosen according to the background data text files.

There are three tables:

run table
sample table
pixel data table

The run table contains the parameters that defind a background run. This are start and stop time, the telescope number, sampling time and some more. For every run a new entry is writen to the database. The sample table gathers data for every sampling time. This means at first the the time stamp and readout timing parameters. The table stores also the mean value of background data distribution over all pixel. The last table stores the pixel data. For every pixel a data set is added to the table with the four background values.

Data types supported by MySQL:

                    signed             unsigned 
tinyint     8 bit   -128...127         0...255
smallint    16bit   -32768 ... 32767   0 .. 65535.
mediumint   24bit   -8388608 ...8388607. 0 .. 16777215. 
int         32bit   -2147483648 .. 2147483647.  0 ... 4294967295.

Value range for the bgrecord variables:

Run table
 `EyeId`  1..4                --> u tiny
 `TSample` 5...30 (evt n *60) --> u small
 `GPSStart`                   --> pc uses  u int
 `GPSStop`                    --> pc uses  u int
 `StatSamples`  1..65536      --> u medium (1 über small)
 `StatOffset`   0...200(vorher 500) --> u tiny/small 
 `NSamples`     12h * 60 * 2 = 1500 / Nacht --> u small

Sample Table
 `TelescopeId` 1..24           --> u tiny,
 `GPSSec`                       --> pc uses u int
 `GPSMiliSec` 0...999ms         --> small
 `TReadout`  0...1000 ms         --> u small
 `TTransfer` 0...1000 ms         --> u small
 `TFile`     0...1000 ms         --> u small
 `TDatabase` 0...1000 ms         --> u small
 `MeanVariance` 3000 ... 200000  --> u medium 
 `VarVariance`                   --> u medium 
 `MeanThreshold` 0 ... 8000      --> u small
 `VarThreshold`  0 ... 8000?     --> u small
 `MeanPedestal`  150000          --> u medium 
 `VarPedestal`   150000?         --> u medium
 `MeanHitrate` 0 .. 1023         --> u small 
 `VarHitrate`  0 ... 1023        --> u small
 `NPixel`      0...440/480       --> u tiny

To create an empty database use the following commands in the mysql administration tool.


grant all privileges on *.* to auger@localhost with grant option;

create database FDKarlsruhe;
create database FDLosLeones;

GRANT ALL ON FDKarlsruhe.* TO auger;
GRANT ALL ON FDKarlsruhe.* TO auger;

use FDLosLeones;

CREATE TABLE `BGRunTab` (
 `BGRunTabId` int(11) NOT NULL auto_increment,
 `DAQRunId` int(11) NOT NULL default '0',
 `EyeId` tinyint(11) unsigned NOT NULL default '0',
 `Filename` char(40) default '',
 `Type` enum('cal','run','test') default 'run',
 `TSample` smallint(5) unsigned NOT NULL default '0',
 `GPSStart` int(11) unsigned NOT NULL default '0',
 `GPSStop` int(11) unsigned NOT NULL default '0',
 `StatSamples` mediumint(5) unsigned NOT NULL default '0',
 `StatOffset` smallint(5) unsigned default NULL,
 `NSamples` smallint(5) unsigned default NULL,
 PRIMARY KEY (`BGRunTabId`)
) TYPE=MyISAM;


CREATE TABLE `BGSampleTab` (
 `BGSampleTabId` int(11) NOT NULL auto_increment,
 `BGRunTabId` int(11) unsigned NOT NULL default '0',
 `TelescopeId` tinyint(2) unsigned NOT NULL default '0',
 `Time` datetime, 
 `GPSSec` int(11) unsigned NOT NULL default '0',
 `GPSMiliSec` smallint(5) unsigned default NULL,
 `TReadout` smallint(5) unsigned default NULL,
 `TTransfer` smallint(5) unsigned default NULL,
 `TFile` smallint(5) unsigned default NULL,
 `TDatabase` smallint(5) unsigned default NULL,
 `MeanVariance` mediumint(5) unsigned default NULL,
 `VarVariance` mediumint(5) unsigned default NULL,
 `MeanThreshold` smallint(5) unsigned default NULL,
 `VarThreshold` smallint(5) unsigned default NULL,
 `MeanPedestal` mediumint(5) unsigned default NULL,
 `VarPedestal` mediumint(5) unsigned default NULL,
 `MeanHitrate` smallint(5) unsigned default NULL,
 `VarHitrate` smallint(5) unsigned default NULL,
 `NPixel` smallint(3) unsigned default NULL,
 `MeanVarianceV` mediumint(5) unsigned default NULL,
 `VarVarianceV` mediumint(5) unsigned default NULL,
 `MeanPedestalV` mediumint(5) unsigned default NULL,
 `VarPedestalV` mediumint(5) unsigned default NULL,
 `NPixelV` smallint(3) unsigned default NULL,
 PRIMARY KEY (`BGSampleTabId`)
) TYPE=MyISAM;


CREATE TABLE `BGPixelTab` (
 `BGSampleTabId` int(11) NOT NULL default '0',
 `PixelId` smallint(3) unsigned NOT NULL default '0',
 `Variance` mediumint(11) default NULL,
 `Threshold` smallint(5) unsigned default NULL,
 `Pedestal` mediumint(11) default NULL,
 `Hitrate` smallint(5) unsigned default NULL,
 PRIMARY KEY (`BGSampleTabId`, `PixelId`)
) TYPE=MyISAM;


CREATE TABLE `ErrorTab` (
 `ErrorTabId` int(11) NOT NULL auto_increment,
 `Time` datetime,
 `GPSSec` int(11) unsigned NOT NULL default '0',
 `EyeId` tinyint(2) unsigned NOT NULL default '0',
 `TelescopeId` tinyint(2) unsigned NOT NULL default '0',
 `SevLevel` enum('info','warn','error','critical','severe','fatal'),
 `Message` text,
 `Process` enum('Esm','Evb','EvbControl','processd','eyerc','feapid','readout','recorder','sender','unknown') ,
 `ErrorSourceTabId` int(11) unsigned, 
 PRIMARY KEY (`ErrorTabId`)
) TYPE=MyISAM;