Document revision history

  • 1.0.3 — July 2024 — Fix typos

  • 1.0.2 — February 2024 — Fix govert option -u description

  • 1.0.1 — January 2024 — Fix typo in curl/wget verbose option in fesh

  • 1.0 — December 2023 — Initial release

1. Introduction

This document covers the steps needed to update from FS 10.1 to 10.2 (latest patch is 10.2.0) and the changes in the new version. Installing the new version (see the Upgrading from 10.1 section below) should be simpler than the update to 10.1 for most users. An appendix, Transferring an existing FS installation to FSL11, provides a suggested path for transferring an existing FS installation to a new FSL11 system, upgrading to 10.2 in the process.

FS 10.2 has been tested for a few configurations but not all possible ones. You should test it carefully before using it for operations. Please email Ed if you encounter problems so that we can resolve them.

The changes in the FS and drudg for 10.2 are covered in the Changes since 10.1 section below. The most significant changes are:

  • Support for FSL11. The FSL11 distribution is based on Debian bullseye and is the latest FSLx Linux distribution. The installation instructions for FSL11 can be found at: https://nvi-inc.github.io/fsl11/.

  • Support for experiment names (session codes) of 16 characters or less. This enables use of the new IVS 12-character session codes.

  • Support for the DBBC3 was extensively improved. It is now roughly complete for the DDC firmware releases available as of June 2023.

  • The display server is now enabled by default. There were also other changes to make it easier to use.

  • The plotlog utility, for examining data in the log files, has been expanded and updated. The changes include support for RDBEs, DBBC3s, and additional data types including recorder performance statistics.

As always, we are deeply indebted to Jonathan Quick (HartRAO) for his many significant contributions that go far beyond what is explicitly mentioned here. For this particular release, his contributions include feature suggestions and testing for DBBC3 support. Those include Tsys filtering, multicast time-out handling, and providing test data for verifying fivpt and onoff for all noise diode configurations with all DBBC3 detector types. Jon also contributed to many non-DBBC3 specific features, testing them, and for documentation enhancements. Many of the improvements that were made are due to his thoughtfulness and diligence.

We thank Beppe Maccaferri (Medicina) for being a diligent tester, reporting bugs, verifying USB vs LSB TPI swaps for DBBC3 Tsys, and providing test data for verifying fivpt and onoff for all noise diode configurations with all DBBC3 detector types.

We thank Eskil Varenius and Jun Yang (both at Onsala) for bug reports and feedback, particularly on DBBC3 related issues.

2. Upgrading from 10.1

You must have already upgraded to 10.1 according to the FS 10.1.0 Update Notes document before installing this update.

Tip
 You can make a fresh install of 10.2 on an FSL11 system using the installation document, https://nvi-inc.github.io/fsl11/. The appendix, Transferring an existing FS installation to FSL11, below in this document, provides a suggested path for transferring an existing FS installation to a new FSL11 system, upgrading to 10.2 in the process.
Note
 Testers of pre-release 10.2 development versions, on any FSLx, can follow the instructions here; some steps may have changed from the development versions. An update notice that specifically covers updating from 10.2-beta1 to 10.2-beta2 and what changed is available at FS 10.2-beta1 to FS 10.2-beta2 Update Notes. An update notice that specifically covers updating from 10.2-beta2 to 10.2 and what changed is available at FS 10.2-beta2 to FS 10.2 Update Notes.

2.1. Backup your operational system

Having a back-up to return to will allow you to continue operations in case something goes wrong with the installation. If you don’t have your own scheme, and for more details, please see the Making a back-up before installing section in the Installation Reference document.

You can also use symbolic links with copies of your old operational directories to switch back and forth between in your operational 10.1 configuration and a test install of 10.2. This is covered in detail in the Using symbolic links subsection of the section and document referenced above.

2.2. Fetch 10.2

There are two alternatives:

  1. If you are using FSL9, FSL10, FSL11, or another system that supports connections to github and you are using git (as is recommended) then as prog execute:

    cd /usr2/fs-git
git fetch
git checkout -q 10.2.0

  2. If you are using FSL8 or are otherwise unable, or prefer not, to use git (you will probably need to be root):

    Refer to the Installing from an archive subsection in the Release Model document. Follow the steps there, through the one that includes the option to set the /usr2/fs link. Please note that:

    • For FSL8, or if you are otherwise unable to connect to github, you will need the TIP in the wget step.

    • For the tag use 10.2.0.

    • You should return to the steps in this document after completing the step setting the link for /usr2/fs by running make install.

2.3. Login as prog

If you are not already logged-in as prog, do that. This is necessary for the remaining steps until Reboot.

2.4. Set new make-time environment variables

There are two alternatives:

  1. For FSL11, no new environment variables need to be set, skip to the next step Make the FS.

  2. For other FSLx Linux distributions, FSL10 and earlier, set the environment variables FS_F95_NO_ALLOW_ARGUMENT_MISMATCH (even if not using f95) and FS_PYTHON_VERSION.

    Note
    		 Some other non-FSLx distributions may need one or both set as well.
    Note
    		 We don’t know if FS versions 10.<x> will work for FSL7 and older distributions as is, but please contact Ed if you run into problems. We expect that they will be easy to fix.

    1. Update login rc files:

      There are two alternatives:

      1. If your prog login shell is bash, add the following to ~prog/.profile:

        export FS_F95_NO_ALLOW_ARGUMENT_MISMATCH=1
export FS_PYTHON_VERSION=2

      2. If your prog login shell is tcsh, add the following to ~prog/.login:

        setenv FS_F95_NO_ALLOW_ARGUMENT_MISMATCH 1
setenv FS_PYTHON_VERSION 2

    2. Logout of prog and back in

2.5. Make the FS

As prog:

cd /usr2/fs
make clean rmdoto rmexe all >& /dev/null
make -s

No output from the last command indicates a successful make.

2.6. Update station code

This step is only needed if your station code uses certain shared memory variables for the schedule, log, or procedure file names. If you do need this step, it will not become necessary until you receive schedule files with more than six characters in the name (before the .skd or .vex extension) or you otherwise want to use .snp, .prc, or .log files with names before the “dot” (.) of more than eight characters. It will probably be some time before schedules with longer names become available. However it is better to not put off updating your software too long. There is a reasonable chance that your software may not need these changes.

Caution
 You may be asked by a coordinating center or scheduler if your station can handle the longer names. You should answer “no” until you have made the necessary changes or verified that no changes are needed.

Click the “Details” toggle below for the full instructions.

Details

There are three issues that may need to be address (as prog): legacy FORTRAN code, shared memory variables, and use of the lognm program.

  1. Legacy FORTRAN code

    If you have FORTRAN code in your station software, you should review the changes in the f95 sub-item (if that link doesn’t work in your browser, click on this link instead: Support FSL11, open the “Details” toggle below that location by clicking on it, go Back in the browser, and finally click on the original link) of the FS changes section to see if any are applicable to your code. Strictly speaking, they are not needed unless you are using FSL11, and of course they can’t be fully tested until you are. As a result, you may want to defer this until you are transitioning to FSL11 (or a later FSLx).

  2. Use of shared memory variables.

    If your station software uses the C shared memory variables: LLOG, LPRC, LSKD, LSTP, LNEWPR, LNEWSK, or LEXPER, you should update it to use LLOG2, LPRC2, LSKD2, LSTP2, LNEWPR2, LNEWSK2 or LEXPER2, respectively.

    Likewise, if you use the corresponding FORTRAN fscom_dum.i variables via the fs_get_variable/fs_set_variable() routines in newlb/prog.c, you will need to change to use the new variables and routines.

    The old variables all have a length of 8 characters. The new variables have a length defined by the MAX_SKD parameter (currently with a value of 18) in include/params.h (C) and include/params.i (FORTRAN).

    Note
    		 The strings in these variables, old and new, are blank padded, not terminated with a null byte.

    The old variables are still available and should work fine until you use .snp, .prc, or .log file names with more than eight characters before the “dot” (.), in which case the values in the old variables will be truncated versions.

  3. Use of the lognm program

    If you use the lognm program, you should make sure the callers can handle log names up to 18 characters in length.

    The lognm program returns a string no longer than the actual length of the log name. There should be no issues for log names of eight characters or less even if you have not made any adjustments to handle longer names.

2.7. Make the station software

The layout of some variables in shared memory has changed. Even if you have not updated your station code, you should re-make your station code to be safe. If /usr2/st/Makefile is set-up in the standard way, you can do this with (as prog):

cd /usr2/st
make rmdoto rmexe all

2.8. Reboot

Important
 This will make sure the FS server is stopped and shared memory is reallocated.

2.9. Login as oper

The remaining steps are to be performed by oper.

2.10. Update control files

  1. Update the comments for rack types in your equip.ctl file.

    Strictly speaking, this is optional, but not updating may cause confusion later if you have a DBBC3, or get one, since there is an additional DBBC3 rack type now. You can use the example/default file in /usr2/fs/st.default/control/equip.ctl for comparison.

  2. Update the dbbc3.ctl control file.

    A line was inserted for the DDC_E firmware version. You can update your file with:

    cd /usr2/control
/usr2/fs/misc/dbbc3ctlfix dbbc3.ctl

    The script will insert the needed line. It will report an error if the file has more, or less, than the number of expected non-comment lines. If that happens, you should correct the file.

    The script will also attempt to update the comments that have changed since FS 10.1. If your file has the original comments from that version they will be updated. If not, or they occurred more than once, a warning will be printed. You may want to look into fixing any discrepancies. You can use the example file, /usr2/fs/st.default/control/dbbc3.ctl, as a guide for making changes.

    Note
    		 The original dbbc3.ctl file will be saved as dbbc3.ctl.bak in case you need to recover.

2.11. Set DBBC3 specific environment variables

There are two alternatives:

  1. If you are not using a DBBC3, no new environment variables need to be set, skip to Remove temporary versions of fesh/plog.

  2. If you are using a DBBC3, you may need to set some environment variables. A reasonable first approach would be to not set any at this point, but you should revisit this issue once you have the new FS installed and otherwise working. A full discussion of the variables can be found in the DBBC3 specific environment variables appendix of the FS DBBC3 Operations Manual. In particular, the section Determining what values to use may be helpful.

2.12. Remove temporary versions of fesh/plog

If you have installed temporary fixes of fesh and/or plog (or other scripts) outside the normal FS source directory, you should probably remove them. All previously supplied updates to work-around various issues have been incorporated into this release.

Typically these temporary versions would have been installed in ~oper/bin and/or have aliases defined for them. We think very few stations have installed temporary versions. If you can’t remember, you can check for an alias, for example for plog, with the command:

alias | grep plog

To check if you are are using a local copy in your PATH you can use the command:

which plog

Instead of just deleting temporary versions or aliases, we suggest you rename them, for example, maybe plog.old for plog. That way they will still be accessible until you have verified that the new standard versions work for you. If you find any fixes are missing, please contact Ed so they can be added.

Note
 If you use a local version of feshp, it should not need to be changed.

2.13. Optional steps

The steps in this section are optional, but you may find them useful. All are to be performed by oper, except as noted. For each item, click the “Details” toggle for the full instructions.

  1. If you can’t use the display server, or prefer not to, disable it.

    Details

    Using the display server is now enabled by default and recommended for all users. However, if you can’t use it (perhaps because it won’t compile), or you prefer not to, you can disable it by setting the environment variable FS_DISPLAY_SERVER to off. Any other value, including on, or the variable not being set at all, will enable it. The steps are:

    Note
    		 If you are unable to compile the display server or have other problems with it, please contact Ed. We will try to resolve them.

    1. Set the environment variable.

      There are two options, depending on what login shell you use:

      • If you use bash, add the following command to ~/.profile:

        export FS_DISPLAY_SERVER=off

      • If you use tcsh, add the following command to ~/.login:

        setenv FS_DISPLAY_SERVER off

    2. Logout and back in.

    3. Update any other accounts that use the FS, usually just prog.

    Caution
    		 This variable should be set the same, or not set, for all accounts/sessions that use fs, fsclient, or (new feature) streamlog. There may be problems if there is a mismatch.

  2. For DBBC(2)/DBBC3 users, set default values for the cont_cal command.

    Details

    If you use continuous cal with a DBBC(2) or DBBC3, you can set the default values for the cont_cal command parameters in your initi procedure. This works for all parameters after the first, state (on/off, or new value undef), which must be set every time (the default is off). The remaining device parameters (polarity, frequency, and option) all default to not being commanded, but will remember a commanded value as the new default. The value commanded for the samples (and filtering for the DBBC3) parameter is also remembered as the default. This allows you to customize the values for your system and use the drudg generated cont_cal commands to control whether continuous calibration, and optionally what polarity, is used.

    You can of course change the defaults after initi has run if you want. This could be useful, for example, if you change receivers. You might want to have a SNAP procedure in the station library for this to minimize the required typing.

    For example, for a DBBC(2), you can set the default polarity, to 2, frequency to 120, an option to 1, and samples to 5 with:

    Warning
    		 This is not intended to be a realistic example. 
    cont_cal=undef,2,5,120,1

    The use of undef prevents any of the device parameters being sent to at this time, but you can use on or off if you prefer and any set values will be sent.

    Similarly, for the DBBC3 you would use:

    Warning
    		 This is not intended to be a realistic example. 
    cont_cal=undef,2,120,1,5
    Caution
    		 The parameter order for the DBBC3 is different from the DBBC(2).

    Please see help=cont_cal for full details on this command for your equipment.

  3. Setting X11 window widths: Increase window size to 146 columns

    Details

    The new default window width for FSL11 of the erchk, login_sh, oprin, and scnch windows is 146 columns. The main advantage of this is that longer log lines can be displayed without wrapping. This includes the standard fixed length error messages, none of which will wrap and should be easier to read. Making this change is recommended if your display can support it. If not, you might adjust the windows to the widest that can be handled conveniently (see the TIP below for using/trying different sizes).

    To update the usual ~/.Xresources file, enter the commands:

    cd
/usr2/fs/misc/xresourcesfix .Xresources

    In .xsession, you should remove the -geometry option (if present) for the login_sh xterm. This will assure that the width is determined solely by the .Xresources file.

    You will need to log-out on the console and log back in to see the full change.

    The script will report an error if it found the geometry value for one of the four windows defined more than once; the extras should probably deleted. A warning will be reported if any of those the geometry value were not found. That may be okay, but may also indicate that the entry was not in the format the script expected. That may need to be looked into.

    Note
    		 The original .Xresources file will be saved as .Xresources.bak in case you need to recover.
    Tip

    The script also includes command line options for setting a different width, either for all the windows or for specific ones and or for not setting the width for specific ones. Enter /usr2/fs/misc/xresourcesfix for the details.

    You can try the script multiple times with different values if between runs you use:

    mv .Xresources.bak .Xresources

    If you have other, special purpose, X11 resource files, you may want to run the script on them also. You may want to make the same changes for prog (and any AUID accounts) as well.

  4. Adjust monit3 and monit7 positions:

    This corrects a previous misplacement of these windows and provides a window title for monit7 that fits in the available space.

    Details

    The previous example .Xresources files accidentally placed the monit3 and monit7 windows so that they slightly overlap the monit2 window. Additionally, the title for the monit7 window did not fit in the available space.

    These changes are recommended unless you have already made adjustments or prefer to keep the windows overlapped to save display space. If you don’t use monit3 (pre-RDBE/DBBC3 Tsys) or monit7 (DBBC3 Tsys), it is still recommended that you make these changes so that you have the default values. The positions of the windows and the title of the monit7 window can be adjusted with the commands:

    cd
/usr2/fs/misc/xresourcesfix2 .Xresources

    You will need to log-out on the console and log back in to see the full change.

    The script will report an error if it found any of the relevant resources were defined more than once; the extras should probably deleted. A warning will be reported if any of the resources were not found. That may be okay, but may also indicate that the entry was not in the format the script expected. That may need to be looked into.

    Note
    		 The original .Xresources file will be saved as .Xresources.bak in case you need to recover.
    Tip

    The script also includes command line options for setting, or not setting, the position of each window, and not updating the monit7 title. Enter /usr2/fs/misc/xresourcesfix2 for the details.

    You can try the script multiple times with different values if between runs you use:

    mv .Xresources.bak .Xresources

    If you have other, special purpose, X11 resources files, you may want to run the script on them also. You may want to make the same changes for prog as well.

  5. Use set revert-all-at-newline on for readline to prevent the bash, oprin, and pfmed histories from being changed.

    Details

    This will prevent history entries (particularly in bash, oprin, and pfmed) from being changed if they are edited and then not used. Their changing in this way can be quite frustrating, but it is up to you if you want to prevent it. In FSL11, the default is to prevent it (this is installed by the make install step for the FS). If you are using FSL10 or FSL9 (but not FSL8), you can prevent it by creating the file, ~/.inputrc with the command:

    cat >~/.inputrc <<EOT
\$include /etc/inputrc
set revert-all-at-newline on
EOT

    You may want to make the same change for prog and root (and any AUID accounts) as well.

    To make this active for:

    • bash — you will need to restart the shell, probably by logging out and back in again

    • oprin — if the FS is running, you will need to restart the FS display client (or restart the FS if you are not using the display server)

    • pfmed — if it is running, exit and restart it

  6. Modernize ~/.Xresources, ~/.xsession, and ~/.fvwm2rc files

    Except for ~/.Xresources, these changes have no material effect on existing installations. The effect for ~/.Xresources is small. Still, you may want to update these files for oper and prog (but not for AUID accounts, which are FSLx specific) to stay consistent with the default. More details about the changes are given in the Cleanup GUI rc files for oper and prog FS change item below (if that link doesn’t work in your browser, click on this link instead: Make miscellaneous enhancements, open the “Details” toggle below that location by clicking on it, go Back in the browser, and finally click on the original link).

    Details

    You may wish to compare your working oper and prog versions of the files to the defaults in the appropriate sub-directories of /usr2/fs/st.default.

    1. Enable pasting with the middle mouse button into all windows, except Operator Input

      In ~/.Xresources remove the resource and value:

      VT100.Translations:       #override\
        ~Ctrl ~Meta <Btn2Up>:   ignore()

      for all windows except oprin.

    2. Modernize ~/.xsession

      • Remove source .profile, but change it to source .login for FSL9 and later if the login shell is tcsh.

      • Remove the -ls option from the login_sh xterm

    3. Modernize ~/.fvwm2rc

      • Remove the -ls option from all xterm invocations

      • Column-align the Operator Menu items so they are easier to read.

      • Add any monit6, monit7, msg/rdbemsg, scnch, and erchk menu, hot-key, and settings that may be missing.

      • Change the menu for the monit7 menu entry to say Monit: DBBC3 instead of Monit: Tsys.

  7. Update your NTP configuration to make it more modern.

    Details

    This change will make the output of the check_ntp procedure and the contents of /etc/ntp.conf file easier to read. Instructions are included for how to remove display of NTP related FQDNs and IP addresses in the log, if that is an issue for your site.

    If you want to make this change, it can be deferred to a convenient time. The instructions can be found in the Update NTP Configuration appendix.

  8. If you use metserver and/or metclient update them to use their new repositories

    Details

    If you use the metserver and/or metclient programs, you may want to update your system to use the new repositories for these programs. If so, this step requires root priveleges. The changes are described in the Remove metserver and metclient programs FS change item below.

  9. If you are using refresh_spare_usr2 with FSL6-FSL9, update the script.

    Details

    If you are using FSL6-9 and have not hit the problem this change is correcting, you may not need it. That, and the fact that there are very few users of this script, is why this change is listed as optional (even though in some sense it is necessary). See the refresh_spare_usr2 FS change below (if that link doesn’t work in your browser, click on this link instead: Make miscellaneous bug fixes, open the “Details” toggle below that location by clicking on it, go Back in the browser, and finally click on the original link).

2.14. Test the FS

Generally speaking, a fairly thorough test is to run a test experiment. Start with using drudg to rotate a schedule, drudging it to make .snp and .prc files and listings. Perform any other pre-experiment preparation and tests that you normally would, then execute part of the schedule, and perform any normal post-experiment plotting and clean-up. The idea here is to verify that everything works as you expect for normal operations.

2.15. Consider when to update your backups

It would be prudent to wait until you have successfully run an experiment or two and preferably received word that the experiment(s) produced good data.

The chances of needing to use your backup from before updating should be small. If you need to restore from your backup and you are using the RAID system of FSL10 or FSL11, you can copy the backup to the (now assumed bad) updated disk. You can then either use the restored disk for operations and/or try the FS update again. The RAID based recoverable testing procedures for FSL10 (https://nvi-inc.github.io/fsl10/raid.html#_recoverable_testing) and FSL11 (https://nvi-inc.github.io/fsl11/raid.html#_recoverable_testing) have more options for recovery. Managing this is a lot easier and safer if you have a third disk.

3. Changes since 10.1

There are separate subsections with summaries of the changes in the FS and drudg.

Clickable links such as #36 connect to specific issues reported at https://github.com/nvi-inc/fs/issues.

A complete history of changes can be found using the git log command from within the FS git working directory, usually /usr2/fs-git.

Each change is listed as a numbered summary typically followed by a “Details” toggle, like:

Details

Details are shown here.

that can be clicked on to toggle showing (or not showing) a collapsible box with the details. In this way, you can view the summary as a list and only reveal the details of items that interest you. The collapsible box may be omitted if providing further details didn’t seem warranted. The collapsible boxes for the details may also have nested collapsible boxes inside them if there are many sub-items.

3.1. FS changes

  1. Support FSL11:

    Details

    The code was updated for FSL11 (based on Debian Bullseye). FSL11 is latest FSLx Linux distribution. The installation instructions can be found at: https://nvi-inc.github.io/fsl11/. The related changes are:

    1. f95: Changes for new f95 compiler version

      Details

      The f95 compiler, version 10.2.1, in FSL11 has stricter requirements for the use of octal, hexadecimal, and binary constants and for argument type agreement in calls to functions and subroutines than in previous FSLx versions. The FS code changes made were:

      1. Reformat octal and hexadecimal constants

        Hexadecimal, octal, and binary constants can only be used in data statements or in the intrinsic function int(). There is a compiler option, -fallow-invalid-boz, to relax this requirement. However, it does not cover the case of actual arguments to a function at this time. There are many of those, so it was decided to just change all non-data statement use of these constants to parameters. They were wrapped in int() in the parameter statements. The parameter names were chosen so the constants could be globally replaced in the FORTRAN code without overflowing the 72-character line limit. Existing octal and hexadecimal constants in parameter statements were wrapped in int(). The FS code did not have any binary constants.

        This change can cause the symbol table for f2c (used by fort77 when it is the compiler) to overflow. The symbol table size for that case was increased by adding the option -Nn1604.

      2. Add use of -fallow-argument-mismatch compiler option.

        Argument type mismatches are common in the code, particular for when Hollerith data is being handled and sometimes interger*4 and sometimes integer*2 arrays are passed as the same argument for functions and subroutines. Since this error is benign and there was an option to ignore it (and it worked), it was used.

        For f95 in previous FSLx versions, this option is not needed, or accepted. To allow compilation on these older systems, use of an environment variable FS_F95_NO_ALLOW_ARGUMENT_MISMATCH was added. If it is set to 1, the complier option is not used. Some other non-FSLx distributions may need the variable set as well. For these older systems, this needs to be set once in prog's login rc files then it is necessary to re-login into the prog account before compiling.

        For FS installations that are using the fort77 complier, it is still recommended that the variable be set to 1. That way the configuration will be forward compatible with a possible change to using f95.

      3. Remove mixing of integer*4 and integer*2 values as arguments for intrinsic and().

    2. Changes for python3.

      Details

      FSL11 has limited support for python2. In particular, there is no numpy. The two largest python programs in the FS, gnplt and logpl, require numpy. Since most of the work converting was for these two programs, it seemed best if they all were converted. There is not expected to by any python2 in the next Debian release, bookworm, and subsequent releases.

      The older python2 versions are still included in case they are needed. For older systems that are unable to use the python3 versions (this includes FSL10 and older FSLx systems), an environment variable FS_PYTHON_VERSION was added. If it is set to 2, the python2 versions are linked. For these older systems, this needs to be set once in prog's login rc files then it is necessary to re-login into the prog account before compiling.

      It may be that on some of these older systems (this includes FSL10 and older FSLx systems), the python3 versions of the scripts could be used if more Debian packages for missing python3 modules are installed. This has not been tested. If you try this, please let Ed know your results. In the meantime, the original python2 scripts are available.

      The programs affected are: logpl, gnplt, monpcal, be_client, s_client, and rdbemsg. The python3 source code is in directories with the same name. The python2 source code can be found in directories with the same name, but with -python2 appended. It is expected that the python2 and python3 versions will be maintained in parallel for the foreseeable future.

      Four steps were used to convert the code to python3:

      1. Run code through the 2to3 utility.

        This utility makes many of the typographic changes needed going from python2 to python3. It can installed, as root, with:

        apt-get install 2to3

      2. Run the code through the reindent utility.

        This utility will re-indent a script using four spaces (and no hard tabs). It can be installed, with:

        1. First, as root:

          apt-get install pip

        2. Then as prog (in bash):

          cd ~
pip3 install reindent
PATH="~/.local/bin:$PATH"
        Tip
        		 The PATH=…​ statement needs to be re-executed for each new login (and previously running bash shell). It can be added to ~/.profile to make it persistent.

      3. Change the shebang lines to use python3

        The python3 variant must be explicitly referenced for FSL11.

      4. Fix runtime issues that were discovered during testing

        We believe we have found all of these, but perhaps not. The most common paths through the code were exercised, but there maybe some obscure paths, particularly in logpl and gnplt that may still have problems. Please report any issues you find to Ed. It should be easy to post (and install) a patch that will fix them.

        The fixes made so far include:

        • Wrap the third argument of range() in int()

        • Change the import from idlelib.TreeWidget to idlelib.tree

        • Replace the use of popen2 with subprocess and use text encoding for sub-process I/O

        • Fix archaic use of strip()

        • Use encode()/decode() for socket I/O

        • Change isAlive() to is_alive() for threads

        • Use key=functools.cmp_to_key() for (time) sort.

        • Set rcond=-1 in linalg.lstsq() to preserve old behavior

        • Use median() rather than divide-by-two to find the median of a sorted list.

      Note

      The installation of 2to3, pip, and reindent can be reversed (if nothing else is installed in ~prog/.local) with:

      1. As root:

        apt-get purge 2to3
apt-get purge pip

      2. As prog (be careful with this command in case other things than reindent are installed in ~prog/.local):

        rm -rf /usr2/prog/.local

    3. New required environment variables for pre-FSL11 systems.

      Details

      Two new environment variables, FS_F95_NO_ALLOW_ARGUMENT_MISMATCH and FS_PYTHON_VERSION, were added to assist with compilation on pre-FSL11 systems. They only need to be set once in prog's login rc files on these systems. Their use is explained in the previous two sub-items.

    4. Remove use of stime()

      Details

      In FSL11, stime() is not available for newly linked applications. It would need to be replaced with clock_settime(). It was not replaced because the functionality it was used for, setting the system time, hasn’t been available to non-root users since the FS was ported to Linux (FS9), circa 1995. The FS must never be run by root.

  2. Names: Support names of 18 characters or less for the schedule, proc, and log commands.

    Details

    With the last two characters of the names usually taken up by the two-character station code, this allows experiment names to be 16 characters or less. This provides support for the IVS Master File v2 “session code” lengths of 12 characters or less.

    This change is largely transparent to the users. The four visible effects are:

    • The length and/or location of the SCHED=…​ and LOG=…​ fields in the System Status Monitor display (monit2) were changed to accommodate the maximum name lengths. The SCHED=…​ label is now SCH=…​ to allow it to fit in the available space and for alignment with LOG=…​. Both are now located in lower-half along the left-hand edge of the window. Some obsolete tape related fields were removed to make this possible.

    • The display of procedure libraries for the pfdl command in pfmed now shows only three libraries per line. A key was also added to describe the prefix letters (>: active, A: schedule, and S: station).

    • The log=…​ command now traps a log name that is too long. Previously, it just truncated longer names to eight characters.

    • The lognm script will put out log names up to 18 characters in length, previously the maximum was eight.

    The fsvue and logex programs were not updated for this change. If you are using these programs, we will look into updating them (please contact Ed). If they were working before, they should continue to work for names of eight characters or less.

    Internally, new variables were appended to shared memory for the schedule, schedule procedure, station procedure, and log file names and the experiment (schedule) name. The old variables are still present and hold up to the first eight characters of the longer names. This is intended to make the changes backward compatible for station programs (such as antcn and telegraf) that are pinned to the FS 10.0 (or 10.1) memory layout until they can be updated for the new variables. Of course, longer names will appear truncated in the downstream programs until they have been updated. The new and old variables are described in the Details collapsible box of the Update station code step above.

  3. Complete DBBC3 support

    Details

    Support for the DBBC3 was extensively improved. It is now roughly complete for the DDC firmware releases available as of June 2023.

    1. Add support for the DBBC3 personality DDC_E

      Details

      This personality is a close analog of the the DDC_U personality, but has better bandpass shapes and is limited to only eight BBCs per Core3H board.

      It can be selected by setting the rack in equip.ctl to dbbc3_ddc_e. The dbbc3.ctl file has an additional line for the version (see the Update control files step above for details on installing this change). Using the string nominal for the BBCs/IF in this file selects a value of eight for this case. The FS DBBC3 Operations Manual was updated to include the new personality.

    2. Adjust handing of TPI data to agree with what was observed in the field (closing #97 and #192).

      Details

      The ordering of TPI data returned by the DBBC3 in the field does not appear to agree with the documentation. Particularly USB/LSB are swapped and in some cases cal-on/cal-off are swapped. The FS handling of the data was adjusted to agree with what was observed. In addition, environment variables were added to allow the handling to be adjusted since there are some variations among firmware releases, particularly early ones, and changes may occur in future firmware releases.

      Environment variables were also added for multicast data to control whether time is expected, to adjust the returned time, and to set how often to report an incorrect firmware version.

      The default values for all these variables are appropriate for the understood cases, but may need to be adjusted for some older firmware versions. The use of these variables are covered in detail in the DBBC3 specific environment variables appendix of the FS DBBC3 Operations Manual.

      Thanks to Eskil Varenius (Onsala) and Beppe Maccaferri (Medicina) for discovering the TPI ordering issue. Thanks to Jon Quick (HartRAO) for providing a test-bed system for detailed exploration of the issue.

    3. Add information to the DBBC3 ifX command help page for the correct target levels.

      Details

      The nominal level is 32000, but needs to be reduced if the BW of the input signal is narrower than the nominal 4 GHz. Approximate target levels for three different input BWs was added. This information was also added to the FS DBBC3 Operations Manual, as the IF target levels section, along with some information on detecting the problem and correcting it.

      Thanks to Jun Yang (Onsala) for finding this issue and pointing out that the compression factor of onoff can be used to identify it. Thanks to Sven Bornbusch (MPIfR) for explaining the cause and providing guidance on reasonable levels. Thanks to Jon Quick (HartRAO) for suggesting using Tsys values as a method for finding the correct level.

    4. Complete support of DBBC3 noncontinuous (legacy) calibration (closing #194)

      Details

      1. Use legacy terminology for noncontinuous calibration

        The documentation now uses legacy calibration as a synonym for noncontinuous calibration, which is explicitly switching the noise diode on and off for Tsys measurements. This usage will eventually extend beyond DBBC3s.

      2. Force the polarity to 0 for legacy Tsys.

        When legacy calibration is used (cont_cal=off,…​), a polarity of 0 is always sent to the DBBC3, regardless of what was requested (even not sending anything). A warning is issued if this overrides the request. This is necessary to place the TPI data in the correct positions in DBBC3 command responses and the multicast data.

      3. Add support for formbbc and formif device mnemonics for DBBC3 Tsys related commands.

        As with with other racks, they refer to the detectors of devices configured for recording: formbbc for BBC channels, formif for IFs that have BBC channels being recorded. They were added for use with tpi, tpical tpdiff, caltemp, and tsys commands. Although DBBC3 legacy calibration was supported in FS 10.0, it was not possible to support these mnemonics until FS 10.1 and the introduction of the core3h_mode command. At that time it was thought (hoped) that legacy calibration would not be needed for the DBBC3.

      4. Include legacy calibration results in the DBBC3 Tsys monitor display window,

        The values of course only update when a new measurement is made. If the commanded equipment setup is not complete enough to define Tcal , Ntcal is shown in inverse video. If no legacy calibration measurements have been made (and continuous calibration is not enabled), N cal is shown in inverse video. Values that exceed 999.9, erroneous values (negative, infinity, and overflow), and other setup errors (BBC or LO not defined), are shown as dollar signs, $$$$$.

      5. Add example legacy calibration procedures for the DBBC3.

        Example caltsys and caltsys_man procedures were added to the st.default/proc/d3fbstation.prc example library.

      Thanks to Beppe Maccaferri (Medicina) and Jun Yang (Onsala) for pointing out support was needed and that formbbc/formif were missing, as well as discovering that the existing legacy calibration support still worked for the DBBC3.

    5. Add averaging and filtering of continuous DBBC3 Tsys data.

      Details

      Averaging was missing from 10.1. It is useful with low power noise diodes and/or varying RFI to provide a more stable Tsys value. Additionally, an experimental filter feature was added for removing RFI affected samples.

      1. Averaging of Tsys data

        This works differently from the DBBC2 continuous calibration averaging which forms an average Tsys value by averaging the TPI counts (as opposed to the Tsys values). That approach gives the highest precision for non-AGC data, but has somewhat reduced accuracy with AGC if there are significant level changes. It is good to keep in mind that Tsys is a station diagnostic and doesn’t have the same accuracy requirements as amplitude calibration, which uses the raw TPI count data.

        For the DBBC3, the averages are formed using exponential smoothing of the Tsys values themselves. The decay time-constant for the smoothing is set, in seconds, by the samples parameter of the FS cont_cal command. Invalid data (Tsys infinity, TPI overflows, and TPI cal-on and/or cal-off values are zero) and Tsys values outside the range ±1000° are discarded. When multicast packets are lost or values are discarded, that is not taken into account; the data is treated as though it had uniform one second spacing.

        The averages are restarted for any of these three events:

        • A change in any FS cont_cal command parameters.

        • The tpicd command is used to reinitialize the sampling configuration. The usual use of this is when a drudg generated mode setup procedure is run, possibly for a mode change.

        • A tpicd=reset command is issued. The reset parameter was added (for DBBC3 racks only) to provide a way to reset the averages manually without otherwise affecting the tpicd configuration. This can be useful if the averages have become skewed due to RFI and/or samples from a different elevation. It can be placed in a SNAP procedure, maybe reset, to minimize typing.

        The cont_cal command samples parameter also sets the number of cycles of the tpicd command for logging the averaged Tsys data. It is expected that the cycle period for tpicd will normally be set to 100 centiseconds, in which case the Tsys values will be logged every samples seconds. If the cycle period of tpicd is set to longer than 100 centiseconds, the logging period will be correspondingly longer than the time-constant.

        The special value of 0 for samples disables averaging and sets the number of cycles of tpicd for logging the Tsys data to one. The logged (and displayed) Tsys values are truly statistically independent only if 0 is used. That was the behavior of FS 10.1.

      2. Filtering of Tsys values

        When averaging is in use, the data cal also be filtered. This can be particularly helpful for bands with significant RFI. Nine new parameters in cont_cal command are used to control filtering (see help=cont_cal or https://github.com/nvi-inc/fs/blob/main/help/cont_cal.j__). In addition to enabling filtering with a filter parameter of 1, the user can specify a threshold, in percent of the average for a device, per IF, for rejecting values from being included in the average. The average is displayed in monit7 with a green background if one or two values have been rejected (clipped) in a row; yellow, three to five; red, six or more. If the average is in the red clipping “zone” and is skewed from an average of the recent values by more than the filtering threshold, it will be auto-reset to the recent average and shown with a blue background. The auto-reset feature should reduce the need to reset manually with tpicd=reset.

        For the next cycle after the auto-reset, the displayed value may clipped, or unclipped, depending on how far the next sample is from the new average.

        It may not be necessary to use different thresholds for different bands. Practical usage of this feature may be to have one set the thresholds high enough, in initi, to clip the outliers in the most affected band(s). Bands with little or no RFI may then still behave normally.

        This feature is experimental and may be changed and other filtering schemes may be added.

      Thanks to Jon Quick (HartRAO) for pointing out that averaging was missing, as well suggesting filtering, the algorithm, colorization, and many fruitful discussions and feedback on the implementation.

    6. Enable tpicd=tsys for DBBC3.

      Details

      Each time this command is used, there is a one-time display of the log entires of the Tsys when continuous calibration is in use. This is useful for getting a static display of the current Tsys values.

      Thanks to Jon Quick (HartRAO) for pointing out that this was missing.

    7. Add not commanding of the polarity, frequency, and option device parameters as an option in the DBBC3 cont_cal command.

      Details

      The behavior is now the same as for the DBBC (i.e., DBBC2). These parameters are not sent to the device unless they have been specified in the FS cont_cal command. Thereafter they are sent with the previously specified value. (A value to -1 can be used to disable sending of the parameter again.)

      Note
      		 As described in the Setting cont_cal defaults change item in this document, it is now possible to use undef as the first (state) parameter to set the defaults for these parameters without sending them to the device.

    8. Improve monit7:

      Details

      1. Replace the unused VDIF Epoch field with multicast centiseconds Arrival time.

        The former was expected to become available in the multicast data, but has not yet been added. Instead the centiseconds, the 0.01 seconds within the second, of the multicast arrival time is shown. This can be useful for diagnosing late transmission, and hence arrival, of multicast packets.

        Generally speaking if the centiseconds is less than 20, the packet from the previous second was late. Usually in that case the packet arrives after the display updates. Data from the previous packet is displayed again, with the Time value in inverse video because it has not changed. In the update for the next second, the data from the late packet is displayed with the low value for the arrival time. The displayed Time from the packet and DBBC3-FS values are increased by one to account for the packet being late. The packet that should have arrived in that second is lost and a multicast time-out occurs. Apparently the previous packet transmission overran the DBBC3 1 PPS, preventing the transmission of the next packet. The Time value will not be inverse again for the missing packet since the late packet was displayed instead.

        The arrival time before which a packet is considered being late (from the previous second) in monit7 can be adjusted with the new command line switch -l (see monit7 -h for help information).

        The VDIF epoch field will be re-added when it becomes available in the multicast.

      2. Add polarization to IF/RF header in the DBBC3 Tsys monitor display (monit7).

        If the LO and its polarization are defined for the displayed IF, the polarization will be shown as (L) or (R).

        Note
        		 (L) or (R) are displayed regardless of what polarization pair is in use: Left/Right, Horizontal/Vertical, or X/Y. Following the usual alphabetical order convention within a pair: LR, HV, and XY, you can assume: L=H=X and R=V=Y. Until the FS is updated to recognize pairs other than Left/Right, you need to know which pair is in use to interpret what is shown.

        Thanks to Jon Quick (HartRAO) for suggesting showing the polarization and the format.

      3. Show negative continuous calibration Tsys values

        Except for clipped (and auto-reset) values, negative values, including out-of-range (dollar signs, $$$$$), are shown in inverse video to highlight that the polarity is wrong. Clipped (and auto-reset) values include a sign.

      4. Invalid continuous calibration data is shown with a cyan background.

        The cases are:

        • If the BBC “on” and/or “off” TPI values overflow (65536), ovrfl is shown.

        • If the “on” and “off” TPI values are zero, tpi=0 is shown.

        • If the “off” TPI value is zero, off=0 is shown.

        • If the “on” TPI value is zero,  on=0 is shown.

        • If the “on” and “off” TPI values are equal (and not zero),  inf  is shown.

      5. Invalid data and out-of-range continuous calibration Tsys values are always shown when they occur.

        This makes it possible to identify error conditions even if averaging (and filtering) are in use.

      6. Add -r option for use with certain reverse video terminals

        Normally, the foreground color for cyan, green, and yellow backgrounds is black; for red and blue, white. It was felt that these combinations were the easiest to read.

        For certain reverse video terminals, the white and black foreground colors are swapped for these backgrounds. For example, this happens if xterm is run with the -rv command line option and the normal FS black/linen settings for foreground/background are in use. To get the unswapped foreground colors, you can use the new -r command line option with monit7 (see monit7 -h for help information). Unfortunately, it is not possible for monit7 to detect the need for this automatically.

      7. Prevent inadvertent escape sequences from causing monit7 from terminating (closing #204).

        Pressing a desktop key combination, such as Ctrl+Alt+↓, that generates an escape sequence, when monit7 had the focus would cause it to terminate. To prevent this, use of Esc to terminate was removed. To prevent other characters in the escape sequence from being treated as monit7 commands, they are filtered according to “CSI sequences” at https://en.wikipedia.org/wiki/ANSI_escape_code. If an out-of-order byte is received, processing of the escape sequence is terminated and the byte is treated as a new input, either another escape sequence or a command. This effectively filters out the user pressing Esc, or key combinations that generate escape sequences, possibly more than once in a row (including automatic repeat).

        Thanks to Jon Quick (HartRAO) for reporting this issue and testing the fix.

      8. N cal is now displayed when continuous calibration is not enabled (and no legacy calibration is available).

        Previously it was Nccal (and only applied to continuous calibration).

      9. Allow more digits for negative Tsys (continuous calibration)

        A larger range of negative values is shown by dropping the fractional digit as needed. The number of significant digits shown is not reduced.

      10. Always clear BBC Tsys fields that aren’t being displayed in monit7.

        In certain cases when changing modes, such as when a particular IF was selected before Def or Rec came into effect, some old non-blank fields were not cleared when they should have been.

      11. Correct mode selecting not working if an IF was selected before Def or Rec mode started.

        Now normal mode switching is always available.

    9. Log negative DBBC3 continuous Tsys values

      Details

      Negative Tsys values occur with continuous calibration data if the polarity is incorrect. The polarity should be fixed, but the values are logged in case they are useful until the situation is rectified. They only difference from the correct values is the sign.

    10. Change the order of commands for DBBC3 syncing.

      Details

      The sequence of commands for syncing the DBBC3 were changed in the Sync time section of the Alternate Core3H board configuration appendix of the FS DBBC3 Operations Manual.

      There is now a pps_sync command both before and after the timesync commands. Despite this improvement using this method is still not recommended.

      Thanks to Sven Dornbusch (MPIfR) for providing the best sequence of commands.

    11. Add DBBC3 ddc as a condition for the if command.

      Details

      Currently only DDC personalities are supported for DBBC3, but this inclusion makes DBBC(2) caltsys procedures for legacy cal compatible with the DBBC3 and will support possible future expansion to other personalities.

    12. Add error message for Core3H boards having different times in the multicast data.

      Details

      If all the boards don’t have the same time, this is reported every 20 seconds until it is fixed.

    13. Switch to adaptive multicast time-out detection

      Details

      Previously the time-out interval was a fixed 125 centiseconds (CS). Now it is 145 CS if the previous read did not time-out. If it did, the interval is adjusted to 100 CS until the next received packet. On the assumption that multicast transmissions are only delayed 25 CS or less, this allows a generous margin for delayed messages while still detecting missing packets. The interval must be less than 150 CS to avoid possibly missing a time-out after a delayed packet that didn’t time-out.

      The previous, fixed, interval of 125 CS was too small given the observed variation in the packet arrival times. It should have been increased to something a little less than 150 CS. However, even that longer fixed interval would not detect the third of three missing packets in a row. The new scheme should detect all missing packets, as well as collect all the available messages.

    14. Change multicast time-out error reports to use a summary format

      Details

      The first occurrence of a time-out is reported. Then a count of the time-outs in the next 60 read attempts is kept. The number is reported (the initial time-out is not included in this count). This continues until there are 60 attempts with no time-outs. That is reported and then keeping a count ends until the next time-out.

      This approach was adopted because in some cases, time-outs come in bursts. It does not increase the number of messages if there is an isolated time-out. It does reduce, but not eliminate, the messages when the firmware is being reloaded and other cases of persistent or repetitive time-outs. If multicast data is being logged with a 100 centisecond interval (tpicd=…​,100), it is possible to reconstruct which messages were lost.

    15. Change the order of TPI values for DBBC3 iftpX commands to on then off

      Details

      This disagrees with the DBBC3 documentation, but agrees with what the DBBC3 returns for polarity 0. It also agrees with the order of all other on and off values in the DBBC3 documentation (for bbcNNN, multicast IF, and multicast BBC data).

  4. Enable the display server by default and make related changes.

    Details

    1. The display server is now enabled by default.

      Details

      This is recommended for all sites. However, if you can’t use it (perhaps because it won’t compile), or prefer not to, you can disable it by setting the environment variable FS_DISPLAY_SERVER to off. Any other value, including on, or the variable not being set at all, will enable it.

      Caution
      		 This variable should be set the same, or not set, for all accounts/sessions that use fs, fsclient, or streamlog, usually just oper and prog. There may be problems if there is a mismatch.

    2. Shutdown display server on terminate (closing #176 and #208).

      Details

      Previously, if the display server was in use, it continued running in background when the FS was terminated; now it will shutdown. Not shutting down was introduced in commit 85b24dc67111d82371c3fd0b850b19174840e0e4, and first released in FS 10.0.0, as part of a larger scheme to serve client web pages. In the short-term, that plan is not being followed through on and the change had some negative impacts for local use. Manually stopping the server was required in certain cases:

      • If antcn, or another local program opens an X11 application, say for example, for a dialog box to let the operator select the antenna, the application will appear on that display. If later an operator on a different display wants to restart the FS, the server would have to be stopped before restarting the FS for the X11 application to appear on the new display.

      • To update the environment variables used by the FS

      • To change the user that owns the FS processes

      Manually stopping the server is no longer required in these, or any other, cases.

      Note
      		 The Runtime variables section of the FS Environment Variables document and the FS 10.0.0 Server changes document were updated to reflect this change.

      Two other modifications were needed to support this change:

      1. Add interlock for autoftp and fs.prompt.

        An implication of stopping the server is that any running autoftp and fs.prompt processes will also be terminated. This is undesirable, especially in the case of autoftp since any active data transfers would be terminated. To avoid this, an interlock was introduced. When the server is in use and any autoftp or fs.prompt instances are active, termination will be prevented with explanatory error messages. If it is necessary to terminate, an override parameter, force, can be used:

        terminate=force

        To keep things simple, the previous override parameter, disk_record_ok, for terminating if disk recording is active has been eliminated and that functionality is now included in the force parameter as well. See help=terminate for more explanation.

        The interlock for preventing termination if pfmed is active was moved to be before the interlocks that can be overridden with force. It is not possible to override the pfmed interlock and there is no point using force if termination will be blocked by pfmed anyway.

      2. Add a variable wait to delay a quick restart of the FS.

        Stopping the FS server when the FS is terminated can cause a problem if the FS is restarted quickly after being stopped; the restart could fail with the error message:

        fsclient.c:436 (fetch_state) error unable to connect to server: Connection refused

        Apparently, it takes the server a couple seconds to shutdown completely so it can be restarted. To avoid this error, a wait of up to two seconds since the previous FS exit is used. If a wait is needed, a message about its length is printed before the wait. After the wait, the FS starts up normally. An interval of two seconds was tested under a variety of conditions and found to be reliable, while not excessive. In case it is not sufficient in some cases, the error message above was augmented with an explanation and a suggestion to try again.

    3. Cleanup fs and fsclient command-line options

      Details

      1. The fs options -b (background) and -f (foreground) have been removed.

        Whether or not the display server is enabled must be the same across all sessions that use fs, fsclient, erchk, and streamlog. The only way to achieve this is by consistent use of the FS_DISPLAY_SERVER environment variable. Thus it no longer makes sense to allow fs to override the session’s setting. Doing so would cause incorrect behavior, even within the session that started the FS.

      2. It no longer makes sense for users to invoke the fsclient option -f (force).

        The server no longer runs when the FS is not in use. This makes it useless to invoke fsclient without the FS running. The option was removed from the help output. The option is still used internally by the FS, but it may be possible to eliminate it entirely.

      3. The warning messages for the fs option -n (--no-x) were improved.

        The messages identify the programs that are not being run when use of X11 has been disabled. The output text is now consistent with the current form of the options.

    4. Add environment variable FS_DISPLAY_SERVER_CONFIRM_TERMINATE.

      Details

      When the display server is in use, a terminate confirmation prompt is included in oprin as a safety mechanism to reduce the chances that the FS will be terminated while another user (client) is using the FS. Although it is not recommended to disable this prompt, if you don’t want it, you can set the variable to off. For all other values, including it not being set, the prompt will be used. The setting of this variable in the session that started the client, with either the fs or fsclient, determines the behavior of oprin for that client.

      Tip
      		 For simplicity, it is recommended that this variable be set the same, or not set, for all accounts/sessions that use fs or fsclient, usually just oper and prog.

  5. Expand and improve plotlog, the utility for plotting log data.

    Details

    This utility, for examining data in the log files, has been extensively expanded and updated. The changes include support for RDBEs, DBBC3s, and additional data types including recorder performance statistics.

    1. Change the plot device for X11 displays to /xw (closing #183).

      Details

      If the DISPLAY variable is set and no other plot device was specified, the program assumes it should plot on the X11 display. The old default X11 plot device, /xterm, didn’t work. That device apparently worked for some pre-FSL8 distribution. For as far back as FSL8 /xterm seems to be available, but doesn’t work. So this has probably been a problem since at least 2009. plotlog was introduced (using /xterm) in FS 9.8.0 (July 2005) with commit 52398939d5f867b2e7ab4e18f8886babda6dfaae. FSL5 (woody) was probably active at that time. /xw now seems to be a good choice in FSL8 and later.

    2. Plot the data in the current log file if the FS is running and no log was specified on the command line.

      Details

      This can useful for plotting data from the currently running schedule.

    3. Add recorder performance statistics

      Details

      Time plots were added for:

      • Delay in recorder starting (seconds)

      • Shortness of recording length (seconds)

      • Missing bytes (count)

      All information is inferred from the scan_name=…​ command, the command that starts the recording (disk_record=on or mk6x=record=…​), and the results of scan_check. The FS, cplane (Mark 6), and jive5ab forms of scan_check are all supported.

      Thanks to Jon Quick (HartRAO) for suggestions about what information to report.

    4. Expand clock plotting.

      Details

      The clock plotting was expanded to plot all data collected by commands with names that contain fmout, gps, and maser. Additionally, RDBE dot2pps and dot2gps data from multicast and dbe_pps_offset and dbe_gps_offset commands are plotted; DBBC3 pps2dot data from multicast and the mcast_time command are plotted. For the RDBE and DBBC3, if both command stream and multicast versions are available, only the multicast is plotted unless the -B option is used, which will include both (plots can be further filtered with the -g/-G options).

      Opposite signed versions of the same offset (e.g. gps-fmout and fmout-gps) are no longer combined in one plot (with appropriately adjusted signs). Keeping them separate makes the plots more representative of the log contents.

      Thanks to Karine Le Bail and Rüdiger Haas (both at Onsala) for arranging to produce experiment logs with mcast_time data for testing.

    5. Add plotting of wind speed and direction.

      Details

      If fields for these data are present in the wx/ log entries, they will be plotted. This is in contrast to temperature, pressure and humidity, which are always plotted if wx/ entires are present. Missing values for plotted fields are shown as “out-of-range” (near the top-edge of the corresponding plot).

    6. Add plotting of CDMS data.

      Details

      As with cable/ data, the default is to plot the values as the change in one-way delay in picoseconds, relative to the first valid value found in the log. Also as with cable/, the -r option can be used to plot the raw values instead. Values greater than 999998.5, which only occurs for error conditions, are marked as “bad”.

    7. Add plotting of RDBE and DBBC3 Tsys values from multicast.

      Details

      By default, only the data from the first encountered detector (other than channel 00 for RDBEs) from each IF band is plotted. The -m, and -M, options can be used to select, and deselect, different sets of detectors based on regular expressions. This is similar in function to the -g/-G options (the latter, formerly the -e option), except -m/-M only apply to RDBE and DBBC3 Tsys data and are applied as they are read-in instead of when they are plotted. This makes them a bit faster since there are typically many values involved.

    8. Add plotting of RDBE phase-cal data from multicast

      Details

      By default, only the first encountered tone from each IF is plotted. The -d, and -D, options can be used to select, and deselect, different sets of tones based on regular expressions. This is similar in function to the -g/-G options (the latter, formerly the -e option), except that -d/-D are only applied to RDBE phase-cal tones and are applied as they are read-in instead of when they are plotted. This makes them a bit faster since there are typically many values involved.

      The -j (Tsys normalization) and -k options are not supported for RDBE phase-cal yet. However, normalization is typically used in the logged data, as set in rdbe.ctl.

      The (new) -v option plots phase differences between tones in the same RDBE IF channel.

    9. Add plotting of the first encountered LSB phase-cal tone per converter for the Mark IV decoder (and K5TS) output.

      Details

      This is in addition to the already supported first encountered USB tone per converter.

      For phase difference plots (options -lanw) when both USB and LSB tones are present, the differences for only one tone per converter are plotted. If USB and LSB is present for an individual converter, the difference between the side-bands is plotted after the differences for pairs of different converters.

    10. Mark phases outside [-180°,+180°] as bad by default.

      Details

      This can useful for Mark IV decoder communication errors. All values can be included with the new -Y option.

    11. Mark clock and cable values outside (-10,10) seconds as bad by default.

      Details

      These are generally not useful values, but can be included if needed with the new -C option. Normally they only occur if a counter is being used and a bad value is returned.

    12. Add support for Wettzell’s style of /rx/ data.

      Details

      The most useful fields for plotting in Wettzell’s /rx/ data are of the form number[units] where number is a floating point number and units is one of dB, dBm, degC, or MHz. By default, plotlog will only plot what seems to be the most interesting of these, which are the degC fields in any record and the dBM fields in the IFxx records (the dBM and MHz fields in the lox records, and the dB fields in the IFxx records, are usually static). The -W option can be used to plot all the number[units] fields.

      It is assumed that only one field of a given units type exists per log entry type. The latter is determined by the first field of the log entry, typically lox or IFxx, for a given x or xx. For example, loa, lob, IFAH, IFAV are all different types for this purpose. If there is more than one field with a given units type in a log entry type, the plot for that type combination will be garbled. As of this writing there are no known cases of this.

    13. Display bad points as open circles and move them slightly off the top (or right) plot edge.

      Details

      Displaying them as open circles makes it clearer that they are different than the “good” points which are closed circles. Moving them slightly off the top (or right) edge improves their visibility and eliminates ambiguity about which plot they are part of in stacked plots.

    14. Always display values that don’t decode at the upper edge of time plots.

      Details

      Previously for some data types, specifically cable, rx, sx, sk and fmout-gps, samples were omitted if they did not decode as floating point numbers. Now they are displayed at the upper edge of the plot, as occurs for other data types, so their presence is visible. The only cases where samples are completely omitted now is when the form of the entry is too garbled to be identified or the command is missing (possibly because it timed-out). These two situations may be noticeable if the plot for a data type is missing entirely or is sparser than expected.

    15. Plotting of phase differences attempts to provide better vertical plot limits if the differences cluster around ±180°.

      Details

      If there is a gap in the phase differences of 180° or more and there is some data in both the bottom and top of the [-180°,+180°] range, the data is adjusted to be around +180°. This doesn’t fix all overly large vertical scales, but it improves the worst ones.

    16. Add support for the giza plot library.

      Details

      The pgperl package provided by some Linux distributions (for example, FSL11) may use the giza plotting library instead of pgplot. Unfortunately, at least some versions of giza are not a fully compatible replacement for pgplot. Several differences have been noticed, so far, in giza version 1.2.0 (which is used by FSL11):

      Note
      		 FSL11 offers an optional non-standard version of pgperl that uses pgplot. Every effort has been made to make that “safe.” However using it is at your own risk. Installing FSL11 will not include it by default. You can find the directions for installing this non-standard package at: https://nvi-inc.github.io/fsl11/installation.html#_install_pgplot_version_of_pgperl.

      • The default line-width is thicker. It appears to actually be what would be line-width 2 in pgplot. It appears that the line-widths are off by one (see the next item as well).

      • Setting the line-width accepts 0, which gives the same width as 1, the minimum, in pgplot. However, line-width 0 causes the plot borders to not appear for device /xw.

      • The closed circle graph marker 17 is significantly less distinct.

      • The open circle graphs markers, symbols 20 through 27 (and some others), have thicker lines than in pgplot. For 20 and 21, it is difficult to make out that they are open.

      • Graph markers are clipped if they are on the edge of a plot instead of allowing them to spill over. This makes them harder to see.

      • Automatic spacing of vertical tick marks is overly dense.

      • Requested horizontal tick spacings are only approximately respected.

      • The environment variables PGPLOT_BACKGROUND and PGPLOT_FOREGROUND for setting the plot colors are not respected. Versions that start with GIZA_ also do not work.

      If the script detects that giza is in use, it will adjust the line-width, except for plot device /xw, and use a larger open circle for “bad” points. The resulting plots are usable, but not as good as with pgplot. These adjustments can be disabled, individually, using the -Z option, if they cause a problem or if a later version of giza has better agreement with pgplot. If giza is not detected, the -Z option can be used for force the adjustments. Please see the -h output for more details.

      One advantage of giza is that a PDF file is available as an output device. Without giza, you can convert a PostScript file to PDF with ps2pdf.

      Marjolein Verkoutor (JIVE) points out that another advantage of giza is that it can produce PNG files.

    17. Fix bad horizontal ticks for -p option.

      Details

      Previously except for the last page, there was an extra set of horizontal tick marks in the bottom plot on each page. Additionally, the horizontal tick labels on these pages were for the extra set of ticks. This has been fixed. There is no extra set of tick marks and the labels are correct.

    18. Change the command line options.

      Details

      In addition to adding the -B, -C, -d/-D, -m/-M, -W, -Y, and -Z options as mentioned above, the following changes were made:

      1. The old -e option was moved to -G (now paired with -g) for parallel construction with -D/-d and -M/-m and to make room for the new -e option.

      2. The new -e option can be used to specify the rack type as dbbc3 or rdbe, This can be useful for DBBC3 and RDBE log snippets that don’t contain an equip line near the start. This only affects DBBC3 and RDBE Tsys, and RDBE phase-cal, processing.

      3. The new -l option can used to specify the location, which is only used in the plot titles. This can be useful for log snippets that don’t contain a location line.

      4. The new -S option can be used to require a leading slash before the command name for wx/, cable/, and cdms/ entries. For example with -S, the search string for wx/ entries is /wx/. This is useful, for example, if there are non-data entries of the form wx/ and the data entries are of the form, /wx/. The program accepts the form without the leading / by default because that is what some stations produce for the data entries and that will match for stations that do use as a leading /. This option is only helpful for stations with non-data entries that do not have the leading / and data entries with the leading /.

      5. The new -T option can used to specify a string to replace the log file name in the plot titles. This can be particular useful if more than one log is used on the command line, resulting in a “merged” plot.

      6. The old -v (version) option was moved to -V to make room for the new -v (VGOS) option, which plots phase differences between phase-cal tones within an RDBE IF.

    19. Trap attempts to use a nonexistent command line option.

      Details

      The script now stops if this occurs instead of continuing with an error message that might be missed.

    20. Improve the -h help output.

      Details

      1. A suggestion for a file name extension for the /vps device was added.

      2. The explanation of the -2 option was improved.

      3. How to set the background and foreground plot colors was added.

        This can be used to change the background/foreground colors to white/black from black/white. The latter are used by default for the X11 display with some FSLx.

      4. An explanation was added that out-of-range phase values in the -p plots are placed near the right-hand edge of the plots.

      5. An explanation of what happens when more than one log file is specified on the command line, i.e., the data from all the logs is merged.

      6. Add explanation of the option philosophy:

        Generally, the philosophy is that if no options are specified the script should do something that is likely to be useful. Options can be added to tune the behavior for different situations. Scripts or aliases can be used if any options are needed routinely.

    21. Bump version number to 2.4.

    22. Improve the code

      Details

      A few internal improvements were made:

      1. The efficiency of finding the location log record was improved by only parsing for it if it has not been found before (and was not specified by -l). As a result, only the first one encountered (or the -l value) is used now.

      2. The help output was changed to a multi-line string for easier maintenance.

      3. The order of options in the Getopts call was alphabetized.

      4. Removing DOS end-of-lines (to help with files that were transferred via machines with such end-of-lines) was improved so that it did not need to be handled in each search string.

  6. Setting cont_cal defaults: Cleanup setting parameter defaults for the DBBC and DBBC3 cont_cal commands.

    Details

    The cont_cal commands are unique among FS commands in that the defaults for most of their parameters are the previous values commanded. This is to simplify drudg's generation of the cont_cal=…​ commands in the setup procedures without it having to know additional details of the station. The concept is that the user can set the value of the parameters drudg doesn’t know about (frequency, option, and samples, and additionally for DBBC3: filter, ifa, …​, ifh) in the initi procedure and then they will be the default for those parameters in the commands that drudg generates.

    With this approach, drudg only needs to set the continuous calibration state (on or off) and optionally, if the state is on, the polarity (0, 1, 2, or 3). If the other parameters change for different receivers, something additional will be required, such as commanding new defaults when there is a receiver change.

    Note

    drudg can be configured to not set the polarity and then that can come from the previous value used by the command.

    The skedf.ctl options for controlling how drudg handles the cont_cal command are cont_cal and cont_cal_polarity.

    Two changes were made to correct omissions that prevented this scheme from working in the most general way:

    1. Change the default value for the samples parameter to be the previous value.

      Due to an oversight, it was not possible to set the default for the samples parameter. As result, if a value other than 10 was used, it had to be edited into each cont_cal=…​ command. This has been fixed so that the default for the samples will be the previous value (initially 10).

    2. Add undef as a value for the state parameter to not command the device.

      There was no way to set the defaults without commanding the device. This can be an issue if there is no correct general choice for the state and/or some values may cause a device problem. Using undef for the state (instead of on or off) allows setting the defaults for the other parameters without commanding the device.

    Thanks to Jon Quick (HartRAO) for pointing out that the implementation was incomplete and fruitful discussions about how to complete it.

  7. Add streamlog utility for receiving FS log and display output (closing #64).

    Details

    The streamlog utility is a new script that outputs log entries as they are written. It can be used by itself or with other programs that filter for specific log entries. It will provide the most complete output when the display server is enabled, but should also be useful when it is not.

    By default, if the FS is already running, the script will output log entries to stdout (for simple interactive use, this is the user’s terminal) as they are generated. A small number of entries may be lost when the FS is started. When the display server is not enabled, a small number of entries may be lost when the active log is changed.

    The script has four command line options. Generally speaking they should not be used with streamlog in stpgm.ctl. The four options are:

    1. -d — display stream

      This option is only available if the display server is enabled. It outputs the display server stream instead of the log stream. The display stream is what is displayed in the log display window by the FS client. There are several differences between what is is shown in the log display window and what goes in the log. The most significant of these are:

      • The log display output uses a shorter time-tag field.

      • Some output lines are suppressed in the log display window because they would be overwhelming and would generally not be helpful for interactive use.

      • Some FS error messages are not shown in the log display window because the operator has suppressed them with the tnx command.

      • The log display window includes some output that is not in the log, specifically the FS startup and termination messages and some program error messages.

    2. -h — help output

    3. -s — scroll-back

      When the display server is enabled and the script is started and/or the FS is started, any log entries in the scroll-back buffer will also be output. This may reduce the number of lines that might be lost when the FS is started.

      If the display server is not enabled, up to 20 (a little more than the number of lines in the typical log header) old log lines will be output when the script is started, the active file log is changed, or the FS is started. This may result in some lines being output more than once. It may reduce the number of lines that will be missed during these transitions.

    4. -w — wait for FS start

      Wait for the FS to start and/or continue to wait for the FS to be restarted if it is terminated.

    Note

    The limitations and considerations for why these options should not be used in stpgm.ctl are:

    • The -d option can be used in stpgm.ctl if the display server is in use. However, it would be safer to use the log output (no -d) instead. If it is used without the display server enabled, it will crash the FS immediately after start-up.

    • The -h option is not useful in stpgm.ctl. Its use will cause the FS to crash immediately after start-up if the display server is not enabled. It will also crash the FS if the display server is enabled and it is used in an n line in stpgm.ctl.

    • The -s option can be used in stpgm.ctl but is of marginal value. It may reduce the number of lines that might be lost at FS start-up. With the display server not enabled, it may reduce the number of lines lost at the transition to a new log.

    • The -w option is not useful in stpgm.ctl and will cause problems in some cases if the display server is not enabled.

    Thanks to Dave Horsley (Hobart) for coming up with the idea for this script, the initial version, and many of the incremental improvements.

  8. Remove metserver and metclient programs:

    Details

    The metserver program serves data from MET3/4/4A meteorological and/or WMT7xx wind sensors connected to serial ports. The metclient program logs data from a server like metserver. They are not part of the FS per se, but were provided with the FS, beginning in 2003, to simplify distribution. They have now been moved to their own publicly accessible repositories, https://github.com/nvi-inc/metserver and https://github.com/nvi-inc/metclient. The instructions in the included INSTALL files set them up independently of the local /usr2/st directory that was used before. This separation makes them more modular and easier to maintain independently from the station FS programs.

    Note
    		 If you have Ethernet-to-serial converters, you can avoid needing serial ports by using gromet, https://github.com/nvi-inc/gromet, instead of metserver.

    When the programs were split out of the FS repository, the commit messages, which often were not specific to these programs, were updated to provide more relevant information.

    There is no need for existing users to update their versions of metserver and metclient. However, it may be beneficial to update since any future improvements will be made using the new repositories.

    Although a complete reinstall is not very difficult, a more limited update is relatively simple. Besides downloading the repositories and make-ing the programs, the scripts used to run the programs will need to be updated. There are two possible procedures. Use the one below as appropriate depending on whether you are using init.d or systemd to run the programs.

    • For systems using init.d (deprecated):

      The string assigned to the DAEMON variable in the /etc/init.d/metserver.sh and/or /etc/init.d/metclient.sh file needs to be updated by removing the string st/. Then the daemons need to be restarted with, as appropriate:

      /etc/init.d/metserver.sh restart
/etc/init.d/metclient.sh restart

    • For systems using systemd (preferred):

      The existing /usr2/st/metserver/metserver_systemd.sh and/or /usr2/st/meclient/meclient_systemd.sh scripts need to be copied to /usr/local/sbin and the string assigned to the DAEMON variable in the new copies updated by removing the string st/. Additionally, the new metserver.service and/or metclient.service files from the new repositories need to be copied over the ones in /etc/systemd/system/. Then the daemon needs to be reloaded:

      systemctl daemon-reload

      and the services restarted, as appropriate:

      systemctl restart metserver
systemctl restart metclient

    Once metserver and/or metclient are working from the new repositories, the old versions, usually in /usr2/st, can be removed.

  9. Make miscellaneous enhancements: Make 20 miscellaneous enhancements, which include adding two new utility scripts.

    Details

    1. Add a new navigation button to msg.

      Details

      Instead of going directly to the Ready form after sending the message, the Stop form will now show a Go to Ready button. This allows the user to change to a new schedule before the Ready form is opened. That obviates the need to reload the form, which might be overlooked, after the new schedule is opened.

      Thanks to Jon Quick (HartRAO) for suggesting this.

    2. Add wx data to rdbemsg.

      Details

      The window (and messages) now includes the meteorological data from the wx command (temperature, pressure, humidity, wind speed and its direction). The Update Values button populates those fields in the window with the latest results from the wx command. The operator should make sure to use the wx command at least once before pressing Update Values for the Ready message. Of course, if it was initially overlooked, a wx command can still be issued and Update Values pressed again. Both the python2 and python3 versions of the script were updated.

      Thanks to Arthur Niell (Haystack) for requesting this change.

    3. Change the warning for a large structure size correction for the onoff command and onoff program to not ring the bell.

      Details

      The error number for the onoff command was changed from -307 to 307; the onoff program, changed from -7 to 7. (There is a warning in both places so that the user does not need to wait until onoff finishes to see it.) The warnings were also expanded to handle four character device mnemonics, which was missing before. The warning will still show-up in sterp and erchk. It can be disabled in the latter using the erchk.ctl control file.

    4. Increase default X11 window size to 146.

      Details

      The new default window widths for the erchk, login_sh, oprin, and scnch windows is 146 columns. The advantages of the new width for all four windows is that it is the smallest that will allow:

      • Standard error messages displayed from the log to fit in the login_sh window without wrapping (a minimum of 141 columns is required for this)

      • Standard error messages displayed with the default largest indentation of four asterisks (plus a space) in the erchk window to fit without wrapping (a minimum of 146 columns is required for this)

      • All four of these windows to have an aligned right edge if their left edges are aligned.

      Of course, it is entirely up to you what widths you want to use for your system and that will depend on your display. A script, misc/xresourcesfix is included that will allow you to adjust the widths for these windows, either all the same, or different for each window. If your erchk.ctl uses more than four characters of indentation, you will need to increase the width correspondingly to avoid wrapping. See the Setting X11 window widths sub-step in the Optional steps above for the details of updating your ~/.Xresources files.

      Thanks to Jon Quick (HartRAO) for encouraging this recognition that more modern systems usually have more display real estate and that it should be used.

    5. Adjust .Xresources for monit3 and monit7.

      Details

      The previous example .Xresources files accidentally placed the monit3 and monit7 windows so that they slightly overlap the monit2 window. Additionally, the title for the monit7 did not fit in the available space. A script /usr2/fs/misc/xresourcesfix2 has been provided to update .Xresources files that are in use. See the Adjust monit3 and monit7 postions sub-step in the Optional steps above for the details of updating your ~/.Xresources files.

    6. Skip detectors from inactive RDBEs in onoff.

      Details

      Previously, if an RDBE wasn’t defined, and therefore was inactive by default, and detectors from it were selected for onoff, there would be a fatal error in the measurements when trying to communicate with the device. This has been changed so that the detectors for the inactive device will skipped with a warning message. This allows an existing setup for onoff to still be useful if an RDBE is temporarily unavailable. (Which RDBEs are active can also be set, among those defined, with the active_rdbes command.)

    7. Improve fivpt Tsys RMS.

      Details

      This was increased by adding, in quadrature, the variation of the raw data raw noise diode “on” data to the previously used “off” data, mapped to temperature units. While the calculation is still not technically correct, this should give an indication of how noisy the raw data were.

    8. Add more digits to Tcal value in the #fivpt#fivept record.

      Details

      Two more digits were added to the value to improve the relative precision for small Tcal values as an aid to forensic analysis.

    9. Accept waiting response from RDBE dbe_data_send? query in fmset.

      Details

      When fmset queries the RDBE to see if data sending needs to be turned off, a previously unexpected response, waiting, can occur. This can apparently happen when the state is off. The code was adjusted to take this into account. Also any other unexpected response will now be displayed in the error when reporting it.

      Thanks to Russ McWhirter (Haystack) for explaining the waiting response.

    10. Add optional verbose output for curl/wget in fesh (closing #201).

      Details

      When the -v option is used in fesh verbose output is enabled to for curl and wget. This can be useful for debugging connection issues. The -v option still includes verbose output for drudg.

      Thanks to Eskil Varenius (Onsala) for suggesting this.

    11. Improve plog

      Details

      1. Add error message for no matching log found for the -l option in plog.

        The previous message was not informative.

      2. Ignore certain log files for the -l option.

        Log file names that only consist of the station code are excluded. Files ending with _full.log are not accepted if the station code is ll unless ll immediately precedes _full.log in the file name.

    12. Add a comment to a recovered log and make two minor related improvements.

      Details

      When closing a log, either because of changing logs or the FS being terminated, the FS checks to make sure the expected file exists in the computer’s file system. If it does not, it copies the currently open log into a file with the correct name. This allows recovery of an open log if the file is accidentally deleted or renamed while the FS is running. The following changes were made:

      1. Add a comment to the end of a recovered log

        A comment about the recovery is added at the end of the recovered log. This can useful for reconstructing what happened. The format of the comment is:

        "ddout recovered log file '/usr2/log/name.log'

        where name is the log recovered.

        Note
        		 If the log was closed and reopened using a single log=…​ command, the added comment may have an out-of-order timestamp compared to the first entries after it.

      2. Improve displayed messages for a log recovery

        The non-log message output were made more consistent for both error and non-errors. All messages start with !! help! ** . Messages for errors now all include a bell character (ASCII 007).

      3. Correct which file descriptor is closed.

        Previously the wrong descriptor was closed after the recovery. That could lead to a benign, but confusing error message.

    13. Improve recommended NTP configuration

      Details

      The recommended NTP configuration can be found in misc/ntp.txt (https://github.com/nvi-inc/fs/blob/main/misc/ntp.txt). The “items” listed in the following descriptions are where the change is covered in that file.

      1. Change the check_ntp procedure to not use the -n option of ntpq

        This allows descriptive names, instead of IP addresses, to be displayed for servers by ntpq. Item 6a.

        The example .prc files were updated to agree.

      2. Make aliases in /etc/hosts for all NTP servers for easier reading of ntpq -p.

        This defines descriptive aliases for ntpq -p to display. Item 6d.

      3. Use aliases in /etc/ntp.conf for easier viewing and maintenance.

        With the aliases defined in /etc/hosts, this avoids the need to use IP addresses, which are harder to recognize. Without the defined aliases, using IP addresses was necessary to avoid problems when there is a DNS outage. Item 2b.

      4. Add information on how to redact server FQDNs and IP addresses from log.

        If site IT policies prohibit public dissemination of FQDNs and IP, this information can be used to keep that information out of the log files, which are often uploaded to publicly accessible servers. Item 6e.

      5. Make other minor wording improvements.

    14. Prevent erchk issues when the display server is not enabled.

      Details

      This is to guard against accidentally starting erchk if the display server is not running. This might happen, for example, if the multiple copies copies of erchk are normally used when the display server is running. Two fixes were made that only affect erchk when the display server is not in use:

      • erchk can no longer be started if the FS is not running

      • Multiple copies of erchk cannot be started.

        Having more than one copy of erchk would cause each copy to only get (different) subsets of the error messages.

      Thanks to Jon Quick (HartRAO) for reporting these issues.

    15. Use set revert-all-at-newline on for readline.

      Details

      This update prevents history entries from being changed if they are edited and then not used. This “feature” of changing the history entries can be quite frustrating, particularly in bash (oprin and pfmed are also affected). It can be disabled in FSL9, FSL10, and FSL11, on a per user basis, by creating the file:

      ~/.inputrc
      $include /etc/inputrc
set revert-all-at-newline on

    16. Cleanup GUI rc files for oper and prog:

      Details

      1. Enable pasting with the middle mouse button for all windows except Operator Input

        Previously, pasting into several xterm windows with the middle mouse button was disabled. This was intended to reduce unexpected results if the middle mouse button was accidentally clicked. It was probably an overly restrictive policy. Instead now, it is only disabled for the Operator Input window. The change was accomplished by removing the resource and value:

        VT100.Translations:       #override\
        ~Ctrl ~Meta <Btn2Up>:   ignore()

        in the ~/.Xresources files for all windows except the Operator Input window. It should still be possible to paste into all windows, including Operator Input, with Shift+Insert.

      2. Remove source .profile from the ~/.xsession files

        It turns out that it is not needed when the login shell is bash. gdm (FSL8 and maybe earlier) and gdm3 (FSL9/10/11 and maybe later) both run ~/.profile) before running ~/.xsession.

        If the login shell is tcsh, other action is required. For gdm (FSL8 and maybe earlier), it will be necessary to setup the same environment in ~/.profile as in ~/.login and keep the two files in sync. This is because gdm appears to interpret ~/.xsession with either sh or bash. For gdm3 (FSL9/10/11 and maybe later), the ~/.xsession file seems to interpreted using the account’s login shell. Thus for this case, source .login can be inserted as the first non-comment line to have it be used to set the environment. Comments were added to default versions to explain what to do.

      3. Remove the -ls option for xterms in ~/.fvwm2rc and ~/.xsession files

        This option runs the shell used by the xterms as a login shell, i.e., it runs ~/.profile or ~/.login as appropriate when started. Since the environment was already set by gdm/gdm3, this was redundant (-ls is still used for the xterm in ~/.xsession files of AUID accounts for a different reason, see farther below in the Details for Update AUID default files for FSL11).

      4. Other ~/.fvwm2rc file cleanup.

        1. The entries in the Operator menu were column-aligned for easier reading.

        2. The Operator menu entry for monit7 was changed to Monit: DBBC3 to make it more specific than Monit: Tsys.

      5. Bring ~/.fvwm2rc-before-FSL9 up-to-date

        1. The changes to the ~/.fvwm2rc files were also made to these files.

        2. Missing menu and hot-key items were added for: monit6, monit7, msg, rdbemsg, scnch, and erchk.

    17. Update AUID default files for FSL11:

      Details

      1. Add logic to AUID ~/.profile files to promote to oper/prog automatically

        At the end, the environment variable AUID_PROMOTE_ACCOUNT is checked. If it is set to oper or prog, then promotion occurs automatically (the AUID account password may still need to be entered). This variable is set by the FSL11 scripts oper_x11 and prog_x11, which then run startx to start X11 on the console. startx does not run ~/.profile (the environment from the AUID account is inherited). The ~/.xsession file for the AUID accounts runs the xterm with the -ls option which triggers running ~/.profile. An exit command is used after the promotion script so that when the promoted session exits, the GUI is shutdown. This last step prevents the AUID account from being active on the GUI after the window manager has changed to the one for the promoted account.

      2. Add default ~/.profile_<SUDO_USER> scripts to ~oper and ~prog

        A script is created for each AUID added. These scripts are run by the ~/.profile scripts for these accounts to allow per AUID user (the sudo user) customization. The default version of these scripts just prints a message as a reminder that it is being run. There is no need for the user to update the script file for their AUID account unless it would be helpful.

      3. Cleanup ~/.fvwm2rc

        The operator menu was greatly simplified and renamed AUID menu. Access to FS program through this menu no longer makes sense. The AUID menu entries were column-aligned for easier reading.

    18. Add script, govert, to calculated \(\mathit{\frac{G}{T}}\) from SEFDs (closing #197).

      Details

      \(\mathit{\frac{G}{T}}\) is a common figure-of-merit used for communications antennas.

      The script misc/govert was added to calculate \(\mathit{\frac{G}{T}}\) from onoff output, using a very simple calculation. It may not meet your accuracy requirements. For extended sources, it can be also be impacted by the crude resolution corrections used by the FS. The formula used is

      \(\mathit{\frac{G}{T}=10 log_{10}\left(\frac{8 \pi k}{\lambda^2 SEFD}\right)}\), where \(\mathit{k}\) is the Boltzmann constant.

      The script takes the name of a FS log with onoff output as an argument. It outputs the SIG and VAL records with a G/T column appended. Note that the values in the SIG records are the RMS scatters of the underlying data propagated to the final quantities, not the sigmas of the means for those quantities. A -u option can be used to map the results to unity on the gain curve. Use /usr2/fs/misc/govert -h for help with the script.

      Thanks to Stuart Weston (Warkworth) for requesting this capability and testing the script.

    19. Make miscellaneous improvements (some internal) to pfmed.

      Details

      The visible improvements are largely making the terminology in program messages related to procedure libraries consistent, but some bugs were fixed too. The internal improvements are mostly to make the handling of FORTRAN character variables in subroutines work for arbitrary length variables passed in as arguments.

      1. The visible changes include:

        • In program messages, the term “active”, as opposed to “open”, is always used for the procedure library that pfmed is currently working on.

        • In program messages, the term “library”, as opposed to “file”, is always used for a procedure library, except for some file oriented error messages. Error messages in boss related to procedure libraries were also made consistent.

        • The FS help command pages for the schedule and proc commands were updated to be consistent with the above terminology.

        • A “key” was added to the end of the pfdl command output to describe the prefix letters before the library names (>, active in pfmed; A, the current FS schedule library; S, the current FS station library, always station). These prefix letters are now displayed correctly.

        • Fix pfst command to trap the “old” library not existing. Previously, it would be created as an empty library.

        • Fix pfst command to allow copying of the library that is the active library in pfmed. This was broken for gfortan which allows a file to be open to only one unit, but worked for fort77 (which uses f2c). It now works independently of the compiler being used. This had previously been fixed for the st command in commit ec03102e02ee2525243dfc3fba57981c6781f139 for FS 9.13.1 in August 2019.

        • Improve detection of the FS being active if it is started while pfmed is running, which is apparently okay. There may still be some race conditions for this situation.

        • Improved the error message for pfmed already being in use.

        • A missing error message for no procedure library being active was restored.

      2. The internal changes include:

        • Making character arguments of subroutine independent of the actual length of the passed variable. This was very helpful for making the change in the procedure library name lengths.

        • Make the lengths of character variables consistent with their usage for procedure names, procedure library names, and file extensions. This was very helpful for making the change in the procedure library name lengths.

        • Improve the code for the ds command. This included fixing character subroutine arguments to be adjustable, removing Hollerith use of character variables, and cleaning-up edge cases for the bubble-sort.

        • Make the same terminology consistency improvements (“active” and “library”) in the code and comments that are visible to the user.

    20. Include rdbe30_mon.py

      Details

      This script, written by Russ McWhirter (Haystack), is very useful for evaluating RDBE functionality. Russ has graciously agreed to allow it to be distributed with the FS to simplify making it available to stations that have RDBEs.

      The (original) python2 version is available as /usr2/fs/misc/rdbe30_mon.py2. The python3 version is available as /usr2/fs/misc/rdbe30_mon.py.

      1. Some of its features are:

        • When started, it opens four windows: Command List, Command Log, Monitor, and Plots. The windows may be closed individually, but closing the Monitor window will cause the program to exit. The default positions of the windows can be set with command line options (see below).

        • Command line options:

          • -h multicast_host

          • -p multicast_port

          • -H RDBE_host

          • -P RDBE_port

          • --command, --log, --monitor, and --plot to set the X11 display geometry of the corresponding windows. Only the position of the window should be set, e.g., +0+0, as the value for the option.

        • An enable/disable plotting checkbox and a Phase-cal offset (MHz) entry box on the Plots window. The plots shown are in order (from the top):

          • Raw data

          • FFT of raw data

          • Histogram of raw data

          • Time domain Extracted PCal (Complex)

          • FFT of Extracted Pcal: Amplitude

          • FFT of Extracted Pcal: Phase

          • Count difference for Tcal: IF0, IF1

        • The commands in the Command List window can be edited. Pressing F1, or right clicking, on a command will cause it to be sent to the RDBE.

        • Files:

          These files are created in the current working directory. The value of <RDBE_ADDR> is the IP address, host name, or alias of the RDBE.

          • rdbe30_monrc_<RDBE_ADDR>.db — holds the geometry of the windows between invocations of the script. Geometry values from command line options override these.

          • rdbe30_mon_cmd_<RDBE_ADDR>.log — holds a record of the commands sent to the RDBE and the responses.

          • rdbe30_mon_dat_<RDBE_ADDR>.log — holds a record of the multicast data received from the RDBE. This file can become quite large.

      2. Installation

        1. Install the matplotlib appropriate for your system’s python version, if not already included. This will need to be done by root.

          • Usually for python2:

            apt-get install python-matplotlib

          • Usually for python3:

            apt-get install python3-matplotlib

        2. Switch to the oper acoount.

          Important
          		 For the remainder of these instructions make sure you are in the oper account, switching if necessary.

        3. Copy the version that is correct for your system (for python2, use .py2 instead of .py) to your ~oper/bin directory.

          cp /usr2/fs/misc/rdbe30_mon.py ~oper/bin

        4. If you are using the display server, place a line for each RDBE in your clpgm.ctl control file. You can refer to the default st.default/control/clpgm.ctl file. For example, for RDBE-A (for python2, use .py2 instead of .py) add a line like:

          mona   d popen 'cd /tmp;rdbe30_mon.py -h 224.0.2.10 -p 20021 -H rdbea 2>&1' -n rdbemona

          Substitute: the correct multicast address (-h), port (-p), and IP address, host name, or alias (-H), for your device. For other RDBEs, copy that line and make appropriate changes (for example for RDBE-B: mona → monb, rdbea → rdbeb, rdbemona → rdbemonb, and change the multicast address and port).

          The cd /tmp in the line causes the script’s files to be written to (and read from) /tmp; so they won’t clutter up other directories. They will also be automatically deleted each time the system is rebooted. You can place them in a different directory if you want to preserve them.

          Tip
          		 You can control the initial placement of the windows by adding the --command, --log, --monitor, and --plot options with appropriate placement geometry values.
          Note
          		 If you use a host name alias for the RDBE, as is recommended and is used in this example (rdbea), it must be defined in /etc/hosts.

      3. Running the script

        Important
        		 The script should not be left running during operations. If the plotting function is enabled, it is CPU intensive.

        If you are using the display server, you can run the script from the operator input window, e.g., for RDBE-A:

        client=mona

        If you aren’t using the display server (and even if you are), you can enter the command manually, create a script, or create an alias to run it.

        Exit the program by closing the Monitor window

      4. The following changes were needed for the python3 version:

        • The first three changes for python3 in the “Details” for the Support FSL11 changes above: running the script through 2to3 and reindent, and changing the shebang line.

        • Change import of NavigationToolBar2TkAgg to NavigationToolBar2Tk

        • Change log file output to buffered

        • Use draw() instead of show()

        • Use data encode()/decode() for socket I/O

        • Select real part of complex array for plotting to eliminate warning

        • Remove use of buffer() to linearise an array.

  10. Make miscellaneous bug fixes: Nineteen small bugs were fixed. All were fairly minor but one could cause a crash if a DBBC2 was in a bad state.

    Details

    1. Fix crashes for DBBC2 communication errors (closing #191).

      Details

      There were two errors in class number handing of communicating with a DBBC2 (the FS refers to the device as a “DBBC”). They occurred in the periodic checking of the DBBC2 personality/version and the VSI clock rate. Crashes only seemed to happen when the DBBC2 is in a bad state, and then after about 35 ch -810 Communication error for  DBBC. errors. There should no longer be any crashes even if the DBBC2 is in the bad state. Rebooting the DBBC2 may fix the bad state. That is a good thing to do since calibration data may be lost and other problems may occur while it is in the bad state.

      These errors were present since the addition of these checks in commits 8e59c7c3502243ea7762c2bfa062a06bad427de2, January 2016, released as FS 9.11.9, and 0fd3f965075ae91ec12768a4a4a2f5911e973824, April 2019, released as FS 9.13.1.

      This fix is also included in patch releases 10.0.1 and 10.1.1.

      Thanks to Eskil Varenius (Onsala) for reporting this problem and testing the fix.

    2. Restore DBBC sync prompt for DBBC/Mark5B in fmset (closing #207).

      Details

      When syncing the Mark 5B, prompting to sync the DBBC first was restored. This had been lost in the merge of the main (pre-10.0.0) branch into the VGOS branch, commit 224cc218b7519c300c11c59aea6391a6da3b122b.

      Thanks to Jon Quick (HartRAO) for testing this fix.

    3. Fix plog to support sending multiple files to BKG (closing #186).

      Details

      Due to a bug, plog was unable to send multiple files to BKG in one invocation. The result was that none were sent. This situation comes up routinely when sending both reduced and _full.log files for VGOS.

      Thanks to Kiah Imai (KPGO) for reporting this and testing the fix.

    4. Correct logging of multicast time for DBBC3 Core3H board 5 (closing #198).

      Details

      In 10.1, the time of the log entry time-stamp was being substituted for the time returned from board 5. This only affected DDC_U (and off-label use of DDC_E) firmware. This would have had no impact if the board time agreed with the log entry time, which it usually would. If the board time was actually wrong, that would have still be recorded by the mcast_time command, which should be in midob, and would have been visible in the Tsys monitor display.

      Thanks to Eskil Varenius (Onsala) for discovering this (while testing with DDC_V v125) and helping verify the extent of the issue. Thanks Jon Quick (HartRAO) for provided a system to use in developing the fix.

    5. Prevent multicast time-out errors from being suppressed after the DBBC3 returns an error (closing #195).

      Details

      This caused time-out errors to be ignored until the next successful DBBC3 communication. This could be induced with a bad low-level DBBC3 command, e.g., dbbc3=junk.

      Thanks to Jon Quick (HartRAO) for reporting this bug.

    6. Fix using a DBBC IF channel as a detector in fivpt and onoff with continuous calibration (closing #190).

      Details

      The DBBC (i.e., the DBBC2) does not provide separate cal-on and cal-off TPIs for an IF detector when continuous calibration is in use. Implementing something useful in the FS for this case had been overlooked. Unfortunately until it was fixed, the FS produced unusable Tant measurements in fivpt and nonsensical Tsys values for those detectors in fivpt and onoff.

      This was improved by (i) internally treating that detector as having a Tcal value of -100 (i.e., assuming there is no noise diode for this detector) and (ii) using the unswitched power. This results in fivpt Tant and estimated peak values being printed in percent of system temperature in fivpt and Tsys for that detector as -100 in onoff. If the Tcal defined in the .rxg file was already negative, that value is used instead.

      If the IF channel is not corrupted by RFI, this makes it usable for pointing measurements. It use should still be avoided for gain calibration measurements for other reasons, primarily having a very broad bandpass, but in some cases not having a center frequency that onoff can calculate accurately. Additionally, the value of the “DBBC IF power conversion factors” in equip.ctl may not have accurate values.

      Thanks to Jon Quick (HartRAO) for reporting the existing poor behavior, pointing out that something useful could be done, and testing the improvement.

    7. Improve fivpt Tant RMS for continuous calibration.

      Details

      The existing calculation was wrong. It was corrected and simplified. Note that the RMS (displayed when averaging is being used) is the scatter of the underlying raw data mapped to temperature units.

    8. Fix a fivpt bug that caused multicast time-out warnings for the DBBC3 when not recording.

      Details

      While locking (and unlocking) the gains, the DBBC2 mode for ddbcn program was used instead of the DBBC3 mode. This disabled the suppression of multicast time-outs due to DBBC3 commands being sent when not recording. This was fixed.

    9. Fix parsing of detectors for Tsys related commands: tpi, tpical, tpzero, tpdiff, tpgain, tpdiffgain, caltemps, and tsys (partially closing #194).

      Details

      There were two problems:

      1. Buffer shortness

        The buffers for parsing the commands were too short to accommodate a list of all the individual detectors for a given rack type. Although detectors are usually specified with mnemonics that specify groups of detectors (such as formbbc and formif), the shortness of the buffers prevented specify an arbitrary list of individual detectors. This was corrected for all rack types. However, the number of possible detectors for a DBBC3 is so large that they can’t all fit in the buffer (1024 characters) that FS uses for communication between programs. Until that is expanded, which it will be for R2DBE support, the maximum number of DBBC3 detectors is limited by that.

      2. Buffer not cleared before reuse

        A buffer used to decode the individual detectors was not properly cleared before being used to decode the next detector. As a result, if a detector was incompletely specified, but was completed by leftover characters in the buffer, no error was detected and an incorrect detector may have been selected. This was corrected for all rack types. This bug could interact with the preceding bug (“Buffer shortness”) to prevent a truncated detector at the end of the buffer from being reported as an error.

      Thanks Jun Yang (Onsala) for reporting these issues.

    10. Fix plotting of data from paired commands in logpl (closing #182).

      Details

      logpl can plot data from paired commands. The first command of a pair (its description in logpl.ctl starts with a $) is associated with the second of the pair (its description ends with $). logpl selects the data to plot based on the first command. The next following instance of the second command has the value to be plotted. This can be useful for situations where one command identifies what is being sampled (e.g., a BBC defined by pcalports=) and the data values come from a second command (e.g., amplitude or phase for a single sideband from decode4/pcal).

      A problem arose if the corresponding second command was missing (perhaps due to a time-out) before the next instance of the first command. In that case, logpl used the next occurring second command, even if the intervening first command identified different data. The result was that data from two different selections may appeared on one plot. That makes a mess.

      This was fixed by invalidating the match of a first command if another instance of it occurs, but with a different string value. This prevents a match on the second command of a pair if the first command of that pair with a different string has occurred since the original first command with the right string.

    11. Prevent DBBC communication errors with the PFB personality from causing a crash for the tpi, tpical, and tpzero commands,

      Details

      The PFB personality is not heavily used and what use it gets does not usually include Tsys measurements.

      The error check was lost in commit 224cc218b7519c300c11c59aea6391a6da3b122b, August 2019, for the merge of the main (pre-10.0.0) branch into the VGOS branch.

    12. Remove redundant class number clearing, which only occurred after an error when setting the AGC in onoff for DBBC2s and DBBC3s.

      Details

      The could potentially have caused clearing of a class number that was already in use for something else. The chances of a problem occurring were pretty low.

      This error was introduced with the initial support for the DBBC2 and DBBC3. For the DBBC2, this was commit f3ee6ea64be008b2bb0ebcbf3f54223b43de8b23, January 2013, released as FS 9.11.2; DBBC3, 19a69540f9f03d4b1035cde4bd097a5babb3ddfe, October 2018, FS 9.12.12.

    13. Demote if and setup_proc commands from being immediate execution commands (closing #189).

      Details

      If entered interactively, they would execute immediately even if there was a time block on the operator command stream. This was not an issue for the schedule stream since that stream doesn’t have the functionality of immediate execution commands. As a result, the old behavior was benign for schedules. It was also of no consequence interactively unless there was a time block.

    14. Improve RDBE communication error reporting in onoff (closing #209).

      Details

      The error messages for RDBE communication errors in onoff are now more informative.

    15. Correct legacy Tsys error messages (overflow, infinity, less than zero) to support four character device names.

      Details

      This supports DBBC3 devices and future devices that may have four character names.

    16. Correct DBBC3 bbc_gain first parameter error messages.

      Details

      Previously they were just the messages for the DBBC (i.e., DBBC2).

    17. Correct the DBBC error message for an incorrect detector device in the tpi, tpical, tpzero, tpdiff, caltemp and tsys commands

      Details

      Previously it was the one for the VLBA rack.

    18. Set the home directory permissions for AUID accounts to 0750.

    19. refresh_spare_usr2: Fix the refresh_spare_usr2 scripts for FSL6-8 and FSL9.

      Details

      Since the scripts use set -e, the fuser -k -m /usr2 command will fail if no user is active on the spare computer /usr2 partition. Since no user should be active on /usr2 when the script is used, it should not succeed. The fix for this is to change the command for FSL6-8 to:

      fuser -k -m /usr2 || :

      For FSL9, it is changed to:

      fuser -k -M -m /usr2 || :
      Note
      		 In the uninstalled script, the line is commented out.

  11. Make seven miscellaneous help page, and other help output, improvements

    Details

    1. Correct description of the -l option in the plog -h output (closing #203).

      Details

      The -l (for “latest”) option, will use the latest log that ends with the station’s two letter code and does not contain either point or station.

      Thanks to Eskil Varenius (Onsala) for helping clarify this.

    2. Improve the help pages for cont_cal for the DBBC and DBBC3.

      Details

      1. Clarify that the DBBCs send the continuous calibration control signal to the receiver.

      2. Clarify that the samples parameter, in addition to setting the averaging interval (or decay constant), also sets the number of tpid cycles for logging the averaged Tsys.

      Thanks to Jon Quick (HartRAO) for the information on the calibration signal and pointing out that the logging interval was not clear.

    3. Improve the help page for cont_cal for the DBBC3

      Details

      The only polarity values that should be used are 0 and 2. The TTL signal levels that correspond to these values are stated.

      Thanks to Sven Dornbusch (MPIfR) for this clarification on what values to use. Thanks to Jon Quick (HartRAO) for the information on the signal levels.

    4. Improve help page for bbc_gain for DBBC2 and DBBC3

      Details

      The usage of the second and third parameters for gainU and gainL, respectively, for setting the levels manually was clarified.

    5. Fix -help command line option for gnplt (closing #184).

      Details

      This option was fixed to provide a synopsis of the command line arguments instead of failing entirely.

      Thanks to Jon Quick (HartRAO) for reporting this error.

    6. Improve logpl help contents for the main screen

      Details

      The description of the three bottom buttons in the Plot Details box was improved. This was primarily to say that the deleting of individual points is with a double right-click instead of a left-click. Other small improvements were made.

    7. Improve help page for the setup_proc command,

      Details

      Minor wording improvements.

  12. Make 11 miscellaneous documentation changes to the web pages, beyond the updates for other functionality changes.

    Details

    1. Add appendix Transferring an existing FS installation to FSL11 to FS 10.2 Update Notes.

      Details

      This fills a gap in that there were instructions for how to update to FS 10.2 on an existing system and how to install FSL11, but there were no instructions for how to transition an existing system to FSL11. In principle, this should be part of installing FSL11, but it was much more cumbersome if included in the “FSL11 Installation” document. Instead, a pointer was added in that document, linking to the new appendix in the “FS 10.2 Update Notes” document. This may eventually be moved to the Installation Reference document.

    2. Update Converting to a 64-bit System document.

      Details

      This document was revised to make it clearer how to use it for just transferring the files from an existing operational FS installation to a new system. Only a subset of the original steps is needed. Several other minor changes were made:

      • Update for FSL11

      • Use root account for transferring files if allowed. If not, use other appropriate accounts.

      • User rsync to preserve ownerships.

      • Add “new system” for logins to make it clear which system to work on.

      • Reorganize to create reference copies of old system directories in a dedicated directory first, then make working copies as appropriate.

      • Turn off write permission for reference copies.

      • Recognize that <version> for /usr2/st-<version> may use other formats than semantic versioning.

      • Turn off write permission for group and other for /usr2/st-<version>.

      • Improve instructions for updating home directories.

      • Copy old /etc, /usr/local/sbin, /usr/local/bin, and reference copies of older systems for reference on the new system.

      • Add explanation for generating an ssh key for scp commands when converting to 64-bit.

      • Fix some typos

    3. Change the naming scheme for feature releases documents and references.

      Details

      The patch level (.0) is now omitted in the file names and references for feature releases.

      This makes the document consistent for use with later patches without having to rename, or generate almost completely redundant, documents for the patches. The document will be updated for the patches, i.e., the new tag and any documentation fixes. Thus users who update to a feature release after patches have been released can use the original link. Brief explanatory update documents, e.g, “10.2.1 Update Notes”, will still be provided for users applying patches. Previous feature releases are referred to without the patch level. For feature releases that have not had their documents renamed for this approach, the patch level is included in the anchor text for the link to be consistent. The feature release tag still includes the .0 patch level.

      In the gh-pages, it may be expedient to provide a symbolic link for a feature release that points to the latest pre-release (for example, 10.2.html → 10.2-beta2.html) until the feature release is finalized. This will allow links in issues, discussions, emails, etc. that are created before the final release to still point to the latest information after the release. What is appropriate will need to be decided on a case-by-case basis.

      This information was added to the Update notices section of the Release Model document.

      The documents for previous feature releases (10.0 and 10.1) may be updated to follow this pattern if there is a need.

    4. Improve FS DBBC3 Operations Manual document.

      Details

      In addition to including the other related changes in this document, several other improvements were made.

      1. Remove “10.1” from the title.

        It now covers 10.1 and 10.2. It is written for 10.2 with NOTEs for the small number of differences for 10.1.

      2. Move information on multicast time-outs to a new section, Multicast time-outs, and expand the description.

      3. Improve the description of the use_setup_proc drudg option in the Minimizing the use of setup procedures appendix.

      4. Improve the description of the threadsuffix procedure in the Thread Procedure appendix.

        1. Add a description of the ds added to the datastream label for the file name and that it stays lowercase all the way to the file name.

        2. Add an explanation that this results in a double ds in the file name, which is intentional (closing #193).

      5. Fix the record = …​ command in the FlexBuff Recorder subsection of the Recorder tuning appendix.

        The second colon was missing. The FS code was correct. This was only a documentation issue.

        Thanks to Marjolein Verkoutor (JIVE) for reporting this.

      6. Describe how multicast message arrival time is logged.

      7. Describe monit7 data being stale if there is a multicast time-out.

      8. Improve description of Time field in monit7

      9. Add more detail on Tsys logging

      10. Improve description of setup for using setcl to set FS time.

      11. Improve the description of trouble-shooting continuous calibration problems.

    5. Add documents FS 10.1.1 Update Notes and FS 10.0.1 Update Notes for new patches releases.

    6. Improve FS 10.1.0 Update Notes document

      Details

      1. Add steps to remove temporary fixes.

        There were some temporary fixes provided to stations, without releasing patches, to deal with various issues. The programs affected were fesh and plog for data center changes, as well as rdbemsg. Instructions to remove the versions were added as a reminder to switch back to using the new standard versions, which include all the fixes.

      2. Allow for default branch already being main.

        It is not necessary to change the default branch to main when preserving the old repository if it was cloned around June 5, 2022 or later. In this case, the default is already main.

      3. Improve the organization of the “Fetch” step.

      4. Remove redundant “Review changes” section.

    7. Add a TIP for managing directory names of FS installations from archives in the Install from an archive subsection of the Release Model document.

      Details

      Making a copy as a new directory before making any local changes can make it easier to track changes and which version is in use.

    8. Add a TIP for how to avoid losing your place in a “Details” toggle for a FS or drudg change list item when following a link (basically: right-click and open a new tab).

      Details

      This was added to the Improve presentation FS change sub-item (if that link doesn’t work in your browser, the TIP is copied below) of the FS 10.1.0 Update Notes document. The TIP added is:

      Tip
      		 To avoid this, right click the link, then open it in a new tab (alternatively, open it in a new window), and then click on that tab. To return to the original document, you can close the new tab or click on the original document’s tab (or close the new window), whatever you prefer.

      This information was also added to the Links to different documents and to the inside of collapsible blocks subsection of the FS Font Conventions document.

    9. Add explanation of how to navigate to a link in a “Details” toggle in a different document, if the browser doesn’t support going to it directly.

      Details

      This was added to the Improve presentation FS change sub-item (if that link doesn’t work in your browser, the text is copied below) of the FS 10.1.0 Update Notes document. The text added is:

      • Links that point into a Details toggle in a different document do not work in all browsers. To help with that, if the relevant text is small it is reproduced within an embedded sidebar block (grey background). Otherwise, a second link to the title of the item with the toggle is provided along with instructions to follow the link (probably opening it in a new tab, or new window, would be best), open the toggle, and then search for the anchor text of the original link.

      This information was also added to the Links to different documents and to the inside of collapsible blocks subsection of the FS Font Conventions document.

    10. Improve the FS Font Conventions document.

      Details

      1. Subsections for the following topics were added:

        • Links to different documents and into collapsible blocks

        • Collapsible blocks

        • Embedded sidebar blocks

        • Nesting blocks

        • List continuation

        • Unordered list markers

      2. The example for inline links was improved.

    11. Improve instructions in making_gh-pages.txt.

      Details

      • Use the current default branch, main.

      • Don’t add .adoc files, that should have already been done

      • Don’t delete .html files, so symbolic links redirecting to renamed documents aren’t lost.

      • Don’t commit and push on main, that should already have be done or the workflow is different anyway.

  13. Make 10 miscellaneous cosmetic changes

    Details

    1. Refactor display of RDBE multicast data in the RDBE monitor window (monit6).

      Details

      Adding the feature, in FS 10.1, of using inverse video for an incorrect RDBE DOT time had the inadvertent side effect of causing the DOT time to flash inverse sometimes when the RDBE attenuators were adjusted. This was apparently caused by the attenuator adjustment delaying the multicast enough to arrive after the RDBE monitor display (monit6) had updated. Thus the previous second time was displayed and in inverse video because it was wrong.

      This was fixed by recording the time of the multicast message arrival and using that to decide if the RDBE time is correct. The display update was moved to the next integer second for consistency. The displayed time is incremented by one second so it will agree with other displayed times, the System Status monitor (monit2) in particular. As a result, the oddness/evenness of the time no longer agrees with that of the IF being displayed. The monit6 updates are now better synchronized with the monit2 updates, instead of being about 200 milliseconds later.

      Thanks to Kiah Imai (KPGO) for reporting the interaction with setting the attenuators.

    2. Change description (title) for cable in the default logpl.ctl file to not include length.

      Details

      logpl plots the raw cable data, not converted to length.

    3. Fix gnplt error message for the GUI crashing to mention gnplt rather than LogPlotter.

    4. Remove more extraneous (..) strings from some error messages.

      Details

      The most common of these were fixed in FS 10.1, but a few more were uncovered. These were due to unnecessary setting of variables in dbbcn, mk5cn, and rdbcn. Additionally, a change was made to boss that may eliminate this issue if there are any remaining instances.

    5. Fix column alignment of the header for the APR lines in onoff.

      Details

      Beginning with the T column header, the rightmost ones were moved one character farther right.

      The misalignment had apparently been there since onoff was rewritten in commit 1b68b90f216412b90c31afca2ab1eb7168602280, October 2002, for release FS 9.5.15,

    6. Prevent monit7 from crashing if the rack type is non-DBBC3.

      Details

      Now it prints a warning and then exits after 10 seconds. monit7 is only for use with a DBBC3.

    7. In erchk, remove a leading blank that was inserted if there was no additional indentation.

      Details

      This makes it possible to have flush left-aligned error messages. That might be helpful, for example, to minimized the required width of the erchk window needed to avoid line-wrapping.

    8. Fix tpicd to get current DBBC3 variables from the local copy of the shared memory data structure rather than directly from shared memory.

    9. Simplify error handling in onoff.

      Details

      The handling of errors was simplified. The reporting of overall errors doesn’t use ip[4]. Consequently, it can be ignored.

    10. Correct comments in include/dbbc3_bbcnn_ds.h.

3.2. drudg changes

drudg opening message date is 2023-02-21.

  1. Make typographic changes for the new f95 version in FSL11.

    Details

    These are similar to changes described in the f95 FS change item (if that link doesn’t work in your browser, click on this link instead: Support FSL11, open the “Details” toggle below that location by clicking on it, go Back in the browser, and finally click on the original link). There were only a few octal constants outside of parameter statements and these were in simple assignment statements. All octal constants were wrapped in int(). There were no and() statements with mixed sized integers. There were no hexadecimal or binary constants at all.

  2. Support input schedule files (.skd and .vex) with up to 16 characters before the dot (.) for the extension.

    Details

    This change was made to agree with Names FS change item above.

  3. Prepend ds to the name (second) parameter in the datasteram=add,{thread},* command in the single-thread-per-file threadsuffix SNAP procedure for VDIF recording.

    Details

    The command now reads: datastream=add,ds{thread},*. This change is intended to be forward compatible with emerging file naming conventions for recorded data. For more information on the threadsuffix SNAP procedure, see the Thread Procedure appendix of the FS DBBC3 Operations Manual document.

  4. Make four miscellaneous small bug fixes and improvements

    Details

    1. Improve skdrut/errormsg.f to avoid problems with bounds checking by not hard-coding the character argument’s length.

    2. Fix skdrut/trimlen.f to prevent problems with bounds checking if the string is blank.

    3. Fix drudg/snap.f to use the experiment name internal to the file rather than from the filename.

    4. Fix drudg/strip_path.f to check for enough space to store the file name.

Appendix A: Update NTP Configuration

Although the use of NTP on the FS computer is strongly encouraged for most situations, this entire appendix is optional.

The recommended NTP configuration is described in misc/ntp.txt (https://github.com/nvi-inc/fs/blob/main/misc/ntp.txt). If you have not implemented it before, you can use the information there to do so.

As of FS 10.2 the recommendations have been improved. If you have previously used those recommendations, this appendix describes how to update your implementation for the improvements. You should review the directions below, and the contents of misc/ntp.txt to which they refer, before deciding what to do. The “items” listed in the following descriptions are where the change is covered in that file.

  1. As root:

    1. Add aliases for all your NTP servers to /etc/hosts. Item 6d.

    2. Change your /etc/ntp.conf file to use the aliases in /etc/hosts. Item 2b.

    3. Restart ntpd.

      For init.d systems, FSL9 and earlier, you can use:

      /etc/init.d/ntp restart

      For systemd systems, FSL10 and later:

      systemctl restart ntp
      Tip
      		 Rebooting is an alternative for restarting NTP on any system.

    4. Verify NTP operation (any user can do this)

      1. Check to make sure you get the expected servers listed by ntpq -p.

        If not, you may need to recheck you definition of aliases in /etc/hosts and their use in /etc/ntp.conf;

      2. Check that the remote for each server is an alias.

        If not, you may need to adjust your /etc/hosts file.

  2. As oper:

    1. Update your check_ntp procedure to not use the -n option of ntpq. Item 6a.

      You can also expand the list of servers that are displayed by adjusting the grep command in the procedure. The details are covered in item 6a.

    2. Optionally, redact FQDNs and IP addresses from the output of check_ntp. Item 6e.

      Some sites may have IT policies that require this, but it may be prudent for everyone to implement it,

    3. Verify the output of check_ntp.

      1. Make sure the remote field is the alias, not the IP address, for each server.

        If not, you may not have removed the -n option from ntpq -np to make it ntpq -p.

      2. If you intended to redact IP addresses from the output, make sure no IP addresses appear, but instead the string REDACTED. You can compare to ntpq -p run from a shell, which will not have REDACTED.

        If you still get IP addresses from check_ntp, then you may need to fix the use of the filter in item 6e.

Appendix B: Transferring an existing FS installation to FSL11

This appendix provides a suggested path for transferring an existing operational FS installation to a new FSL11 system. It starts with transferring your files, then you will need to make incremental changes to your files for each critical FS update since your existing version. In the end, you will have version 10.2.

Making the changes on the new system is recommended to avoid possibly compromising your operational system. The only testing you will be able to do on the new system will be after applying the 10.2 update. You can however, apply these updates to your old system before transferring the files. That would allow incremental testing of the updates. Some testing on your FSL11 system will still be required in the end. See the subsection titled Version 10.2 below for more information for the final steps on the new system for this alternate approach.

B.1. Transfer files

A suggestion for transferring your files can be found in the document: Converting to a 64-bit System. You can of course use other methods but if so, the document may still be useful as an outline of what needs to be done. If you use the method in the document:

  1. You should use all steps from Login as root on the new system through Customize files on the new system.

  2. If your new system is 64-bit and you have not already updated your code for 64-bit, you should use the steps Login as prog on the new system and Update station programs for 64-bit.

  3. Skip all remaining steps. However, if your local station programs only uses C, you can test whether it will compile (as prog) with the Make local software step. FORTRAN code may also compile and load at this point, but it may need further changes for FSL11 and FS 10.2.

From here jump to the version specific subsection below that corresponds to your existing system’s FS:

B.2. Incremental changes for specific versions

These changes are applied incrementally to bring your files up to date. For a given existing version you will need to apply the specific update and all that follow.

B.2.1. Version 9.11.<x> or earlier (or FS 9.13.0-9.13.1)

  1. If you have a one of these versions, you should upgrade your local files to 9.13.2 first, please refer to the Upgrading from FS versions before the previous stable section in the Installation Reference document for more information. Afterwards, your station programs may not compile, load, or run successfully until you have applied more updates. Please contact Ed if you have questions or problems.

  2. Proceed to the next sub-step, for old version 9.13.2

B.2.2. Version 9.13.2 (or 9.12.10-9.12.13)

  1. For existing system FS version 9.13.2 (or 9.12.10-9.12.13), first update your local files and software to 10.0 according to Upgrading from an FS9 version in the FS 10.0.0 Update Notes document. The steps need to be modified slightly to take into account that you have an initial install on FSL11 already:

    1. Begin with the step Login as prog.

    2. Unless your new system is 32-bit (not recommended) and you plan to use fort77 as your FORTRAN compiler (additionally not recommended), skip the steps Set FORTRAN compiler and Make the FS.

      If you are using fort77, which must be on a 32-bit system, you should use those steps.

    3. Be sure to use step Update station programs.

    4. If you local station software is C only, you can check if it will compile with the step Make local software, but this is not necessary at this point.

      FORTRAN code may also compile at this point, but it may need further changes for FSL11 and FS 10.2, which are handled in subsection Version 10.1 below.

      You will not be able to test your local software until you have complete upgrading through at least 10.2.

    5. Skip the Reboot step, but it should be benign if used.

    6. Use the steps beginning with Login as oper through Miscellaneous FS related changes.

      Part of the last step references FSL10, but the changes are still relevant.

  2. Proceed to the next sub-step, for old version 10.0.

B.2.3. Version 10.0

  1. For existing system FS version 10.0, update your local files and software to 10.1 according to Upgrading from 10,0.0 in the FS 10.1.0 Update Notes document. The steps need to be modified slightly to take into account that you have an initial install on FSL11 already:

    1. Use step: Login as oper.

    2. Use step: Remove temporary fixes.

    3. Use step: Local customizations.

    You will not be able to run the FS at this point to verify the format of the .rxg files. You will need to do that after you have completed the upgrade to 10.2.

  2. Proceed to the next sub-step, for old version 10.1.

B.2.4. Version 10.1

  1. For existing system FS version 10.1, update your local files and software to 10.2 according to Upgrading from 10.1 in the current document (FS 10.2 Update Notes). The steps need to be modified slightly to take into account that you have an initial install on FSL11 already:

    1. Begin with the step Login as prog.

    2. Skip the steps: Set new make-time environment variables and Make the FS.

    3. Use step Update station code.

    4. Update station code for FSL11:

      There are two FSL11 station code issues that are not addressed in the previous step for updating station code. Specifically, the issues are:

      • Changes in the FORTRAN compiler that may prevent legacy station code from compiling

      • Limited support for python2. In particular, code that depends on numpy will not work without converting to python3.

      If either or both of these issues affect you, please refer to change item Support FSL11, in the main document above, for the changes that were made in FS 10.2 to handle these issues. You can make similar changes for your station software. Please contact Ed if you have questions or problems.

    5. Use all the remaining steps beginning with Make the station software.

      You may need to fix your station code so that it will compile and load and debug it as part of testing the FS. Converting to 64-bit and upgrading to 10.0, 10.2, and FSL11 may have introduced some changes that will only be possible to check now.

B.2.5. Version 10.2

For existing system FS version 10.2, in principle nothing needs be changed in your local files and software. There however may be station code updates that you not have applied yet:

  • If you had been using it on a 32-bit system you may not have made the changes for 64-bit referred to in the Transfer files section of this appendix. If your new system is 64-bit, those changes will be needed now.

  • Likewise, if were using FSL10 (or an earlier FSLx), the changes in the Update station code for FSL11 sub-step in the Version 10.1 subsection in this appendix may not have been applied.

In either and both cases, you will need to apply those changes and make the software afterwards.

In any event, you should still test the FS.