FS 10.2-beta2 to FS 10.2 Update Notes

1. Introduction

This document covers the steps needed to update from FS 10.2-beta2 to 10.2 and the changes in the new version. Installing the new version (see the Upgrading from 10.2-beta2 section below) is simpler than the update to 10.2-beta2.

FS 10.2 has been tested for a few configurations but not all possible ones. You should test it carefully before using it for operations. Please email Ed if you encounter problems so that we can resolve them.

The changes in the FS and drudg for 10.2 are covered in the Changes since 10.2-beta2 section below. The most significant changes are:

Verify fivpt and onoff for all noise diodes configurations with all DBBC3 detector types.
Move position of schedule and log names in the monit2 window.
Improve fivpt, particularly the RMSs for continuous calibration with averaging.

2. Upgrading from 10.2-beta2

You must have already upgraded to 10.2-beta2 according to the FS 10.2-beta2 Update Notes document before following these instructions.

2.1. Fetch 10.2

There are two alternatives:

If you are using FSL9, FSL10, FSL11, or another system that supports connections to github and you are using git (as is recommended) then as prog execute:
```
cd /usr2/fs-git
git fetch
git checkout -q 10.2.0
```
If you are using FSL8 or are otherwise unable, or prefer not, to use git (you will probably need to be root):

Refer to the Installing from an archive subsection in the Release Model document. Follow the steps there, through the one that includes the option to set the /usr2/fs link. Please note that:
- For FSL8, or if you are otherwise unable to connect to github, you will need the TIP in the wget step.
- For the tag use 10.2.0.
- You should return to the steps in this document after completing the “Installing from an archive” procedure with the step setting the link for /usr2/fs by running make install.

2.2. Login as prog

If you are not already logged-in as prog, do that. This is necessary for the remaining steps until Reboot.

2.3. Make the FS

As prog:

cd /usr2/fs
make clean rmdoto rmexe all >& /dev/null
make -s

No output from the last command indicates a successful make.

2.4. Reboot

Important

This will make sure shared memory is reallocated.

2.5. Login as oper

The remaining steps are to be performed by oper.

2.6. Required changes

The steps in this section are required. All are to be performed by oper, except as noted.

If you have a FlexBuff recorders, update local SNAP procedures that call checkmk5.

In 10.1, for FlexBuff recorders, use of checkmk5 was replaced with checkfb. Initial versions of the update notes for 10.1 neglected to mention that if you have local procedures that call checkmk5, you would need to update them to use checkfb instead. If you also have a Mark 5 recorder, you may want to create new versions that call checkfb instead for use with the FlexBuff. You should make the appropriate changes now if you haven’t already.

2.7. Optional steps

The steps in this section are optional, but you may find them useful. All are to be performed by oper, except as noted. If it is present, click the “Details” toggle for the instructions.

Adjust .Xresources for monit3 and monit7.

This corrects a previous misplacement of these windows and provides a window title for monit7 that fits in the available space.

Details

The previous example .Xresources files placed the monit3 and monit7 windows so that they slightly overlap the monit2 window. Additionally, the title for the monit7 window did not fit in the available space.

These changes are recommended unless you have already made adjustments or prefer to keep the windows overlapped to save display space. If you don’t use monit3 (pre-RDBE/DBBC3 T_sys) or monit7 (DBBC3 T_sys), it is recommended that you make these changes so that you have the default values. The positions of the windows and the title of the monit7 window can be adjusted with the commands:

cd
/usr2/fs/misc/xresourcesfix2 .Xresources

You will need to log-out on the console and log back in to see the full change.

The script will report an error if it found any of the relevant resources were defined more than once; the extras should probably deleted. A warning will be reported if any of the resources were not found. That may be okay, but may also indicate that the entry was not in the format the script expected. That may need to be looked into.

Note	The original .Xresources file will be saved as .Xresources.bak in case you need to recover.

Tip

The script also includes command line options for setting, or not setting, the position of each window, and not updating the monit7 title. Enter /usr2/fs/misc/xresourcesfix2 for the details.

You can try the script multiple times with different values if between runs you use:

mv .Xresources.bak .Xresources

If you have other, special purpose, X11 resources files, you may want to run the script on them also. You may want to make the same changes for prog as well.

Remove any -geometry option that may be on the login_sh xterm in ~/.xsession.

This allows this setting for the xterm to be determined by the .Xresources file for simplicity. This is particularly useful if you change the default widths of the xterms.
Modernize ~/.Xresources, ~/.xsession, and ~/.fvwm2rc files

Except for ~/.Xresources, these changes have no material effect on existing installations. The effect for ~/.Xresources is small. Still, you may want to update these files for oper and prog to stay consistent with the default. More details about the changes are given in the Cleanup GUI rc files for oper and prog FS change item below.
Details
You may wish to compare your working oper and prog versions of the files to the defaults in the appropriate sub-directories of /usr2/fs/st.default.
1. Enable pasting with the middle mouse button into all windows, except Operator Input
  
  In ~/.Xresources remove the resource and value:
  
  VT100.Translations: #override\ ~Ctrl ~Meta <Btn2Up>: ignore()
  
  for all windows except oprin.
2. Modernize ~/.xsession
  
  Remove source .profile, but change it to source .login for FSL9 and later if the login shell is tcsh.
  
  Remove the -ls option from the login_sh xterm
3. Modernize ~/.fvwm2rc
  
  Remove the -ls option from all xterm invocations
  
  Column-align the Operator Menu items so they are easier to read.
  
  Add any monit6, monit7, msg/rdbemsg, scnch, and erchk menu, hot-key, and settings that may be missing.
  
  Change the menu for the monit7 menu entry to say DBBC3 instead of Tsys
Update metserver and/or metclient

If you use the metserver and/or metclient programs, you may want to update your system to use the new repositories for these programs. If so, this step requires root priveleges. The changes are described in the Remove metserver and metclient programs FS change item below.

3. Changes since 10.2-beta2

There are separate subsections with summaries of the changes in the FS and drudg.

Clickable links such as #36 connect to specific issues reported at https://github.com/nvi-inc/fs/issues.

A complete history of changes can be found using the git log command from within the FS git working directory, usually /usr2/fs-git.

Each change is listed as a numbered title, usually with a few summary sentences, followed by a “Details” toggle, like:

Details

Details are shown here.

that can be clicked on to toggle showing (or not showing) the details. In this way, you can view the summary as a list and only reveal the details of items that interest you. The summary sentences and/or the details toggle may be omitted if they would not add any new information, usually because it is already covered in the numbered title item, summary, and/or the details are very brief.

3.1. FS changes

Verify fivpt and onoff for all noise diodes configurations with all DBBC3 detector types.

Details

The various DBBC3 swaps of TPI data for USB vs LSB and “on” and “off” are apparently handled correctly for the latest DDC firmware versions available as of June 2023. Some older firmware versions (and possibly some subsequent ones) may not be properly supported. There are some tools to adapt FS operation to take some differences into account. They are discussed in the Swaps of TPI and gain values section of the DBBC3 specific environment variables appendix of the FS DBBC3 Operations Manual. Additional tools can be provided if these are not sufficient.

Thanks to Beppe Maccaferri (Medicina) and Jon Quick (HartRAO), who in yeoman efforts, provided the needed test data.
Move position of schedule and log names in the monit2 window.

They are now displayed together at the left-edge in the lower-half of the window.

Details

In the pre-releases of 10.2, the new, longer, schedule and log name fields were visually separated. This was awkward. To make the schedule name fit in the available space, the label SCHED= was reduced to SCH=. This also improved the vertical alignment of the fields. Some obsolete tape related fields were removed to make this possible.
Improve fivpt

Correct T_ant RMS for continuous calibration. Improve T_sys RMS. Add more digits to T_cal value in the #fivpt#fivept record. Fix a bug that caused multicast time-out warnings for the DBBC3 when not recording.
Details
1. Correct T_ant RMS for continuous calibration.
  
  The existing calculation was wrong. It was corrected and simplified. Note that the RMS (displayed when averaging is being used) is the scatter of the underlying raw data mapped to temperature units.
2. Improve T_sys RMS.
  
  This was increased by adding, in quadrature, the variation of the raw data raw noise diode “on” data to the previously used “off” data, mapped to temperature units. While the calculation is still not technically correct, this should give an indication of how noisy the raw data were.
3. Add more digits to T_cal value in the #fivpt#fivept record.
  
  Two more digits were added to the value to improve the relative precision for small T_cal values as an aid to forensic analysis.
4. Fix a bug that caused multicast time-out warnings for the DBBC3 when not recording.
  
  While locking (and unlocking) the gains, the DBBC2 mode for ddbcn program was used instead of the DBBC3 mode. This disabled the suppression of multicast time-outs due to DBBC3 commands being sent when not recording. This was fixed.
Improve msg

Instead of going directly to the Ready form after sending the message, the Stop form will now show a Go to Ready button.

Details

This allows the user to change to a new schedule before the Ready form is opened. That obviates the need to reload the form, which might be overlooked, after the new schedule is opened.

Thanks, to Jon Quick (HartRAO) for suggesting this.
Improve rdbemsg

The wx data are now included in the window and messages.

Details

The window (and messages) now includes the meteorological data from the wx command (temperature, pressure, humidity, wind speed and its direction). The Update Values button populates those fields in the window with the latest results from the wx command. The operator should make sure to use the wx command at least once before pressing Update Values for the Ready message. Of course, if it was initially overlooked, a wx command can still be issued and Update Values pressed again. Both the python2 and python3 versions of the script were updated.

Thanks to Arthur Niell (Haystack) for requesting this change.
Change the warning for a large structure size correction in onoff command to not ring the bell.

Previously this change was made for the warning issued by the onoff program. That warning was also expanded to handle four character device mnemonics. These changes have now been made for the warning issued by the onoff command itself.
Adjust .Xresources for monit3 and monit7.

Position the monit3 and monit7 windows so they don’t overlap the monit2 window. Adjust the monit7 window title to fit in the available space.

Details

The previous example .Xresources files placed the monit3 and monit7 windows so that they slightly overlap the monit2 window. Additionally, the title for the monit7 did not fit in the available space. A script /usr2/fs/misc/xresourcesfix2 has been provided to update .Xresources files that are in use.
Cleanup fs and fsclient command-line options

The fs options -b and -f have been removed. It no longer makes sense for users to invoke the fsclient option -f. The warning messages for the -n (--no-x) option were improved.
Details
1. The fs options -b and -f have been removed.
  
  Whether or not the display server is enabled must be the same across all sessions that use fs, fsclient, erchk, and streamlog. The only way to achieve this is by consistent use of the FS_DISPLAY_SERVER environment variable. Thus it no longer makes sense to allow fs to override the session’s setting. Doing so would cause incorrect behavior, even within the session that started the FS.
2. It no longer makes sense for users to invoke the fsclient option -f.
  
  The server no longer runs when the FS is not in use. This makes it useless to invoke fsclient without the FS running. The option was removed from the help output. The option is still used internally by the FS, but it may be possible to eliminate it entirely.
3. The warning messages for the -n (--no-x) option were improved.
  
  The messages identify the programs that are not being run. The output text is now consistent with the current form of the options.
Add a comment to a recovered log and make two minor related improvements.

A comment is now added at the end of a log that was recovered. The messages printed to the display in the recover process were improved. The file descriptor closed was corrected.
Details
When closing a log, either because of changing logs or the FS being terminated, the FS checks to make sure the expected file exists in the computer’s file system. If it does not, it copies the currently open log into a file with the correct name. This allows recovery of an open log if the file is accidentally deleted or renamed while the FS is running.
1. Add a comment to the end of a recovered log
  
  A comment about the recovery is added at the end of the recovered log. This can useful for reconstructing what happened. The format of the comment is:
  
  "ddout recovered log file '/usr2/log/name.log'
  
  where name is the log recovered.
  
  Note
  If the log was closed and reopened using a single log=… command, the added comment may have an out-of-order timestamp compared to the first entries after it.
2. Improve displayed messages for a log recovery
  
  The non-log message output were made more consistent for both error and non-errors. All messages start with !! help! ** . Messages for errors now all include a bell character (ASCII 007).
3. Correct which file descriptor is closed.
  
  Previously the wrong descriptor was closed after the recovery. That could lead to a benign, but confusing error message.
Remove metserver and metclient programs :

They were moved to separate repositories.
Details
The metserver program serves data from MET3/4/4A meteorological and/or WMT7xx wind sensors connected to serial ports. The metclient program logs data from a server like metserver. They are not part of the FS per se, but were provided with the FS, beginning in 2003, to simplify distribution. They have now been moved to their own publicly accessible repositories, https://github.com/nvi-inc/metserver and https://github.com/nvi-inc/metclient. The instructions in the included INSTALL files set them up independently of the local /usr2/st directory that was used before. This separation makes them more modular and easier to maintain independently from the station FS programs.

Note
If you have Ethernet-to-serial converters, you can avoid needing serial ports by using gromet, https://github.com/nvi-inc/gromet, instead of metserver.

When the programs were split out of the FS repository, the commit messages, which often were not specific to these programs, were updated to provide more relevant information.

There is no need for existing users to update their versions of metserver and metclient. However, it may be beneficial to update since any future improvements will be made using the repositories.

Although a complete reinstall is not very difficult, a more limited update is relatively simple. Besides downloading the repositories and make-ing the programs, the scripts used to run the programs will need to be updated. For systems using init.d (deprecated), the string assigned to the DAEMON variable in the /etc/init.d/metserver.sh and/or /etc/init.d/metclient.sh file needs to be updated by removing the string st/. Then the daemons need to restarted with, as appropriate:
/etc/init.d/metserver.sh restart /etc/init.d/metserver.sh restart
For systems using systemd (preferred), the existing /usr2/st/metserver/metserver_systemd.sh and/or /usr2/st/meclient/meclient_systemd.sh scripts need to be copied to /usr/local/sbin and the string assigned to the DAEMON variable in the new copies updated by removing the string st/. Additionally, the new metserver.service and/or metclient.service files from the new repositories need to be copied over the ones in /etc/systemd/system/. Then the daemon needs to be reloaded:
systemctl daemon-reload
and the services restarted, as appropriate:
systemctl restart metserver systemctl restart metclient
Once metserver and/or metclient are working from the new repositories, the old versions, usually in /usr2/st, can be removed.
Cleanup GUI rc files for oper and prog :
Details
1. Enable pasting with middle mouse button for all windows except Operator Input
  
  Previously, pasting into several xterm windows with the middle mouse button was disabled. This was intended to reduce unexpected results if the middle mouse button was accidentally clicked. It was probably an overly restrictive policy. Instead now, it is only disabled for the Operator Input window. The change was accomplished by removing the resource and value:
  
  VT100.Translations: #override\ ~Ctrl ~Meta <Btn2Up>: ignore()
  
  in the ~/.Xresources files for all windows except the Operator Input window. It should still be possible to paste into all windows, including Operator Input, with Shift+Insert.
2. Remove source .profile from the ~/.xsession files
  
  It turns out that it is not needed when the login shell is bash. gdm (FSL8 and maybe earlier) and gdm3 (FSL9/10/11 and maybe later) both run ~/.profile) before running ~/.xsession.
  
  If the login shell is tcsh, other action is required. For gdm (FSL8 and maybe earlier), it will be necessary to setup the same environment in ~/.profile as in ~/.login and keep the two files in sync. This is because gdm appears to interpret ~/.xsession with either sh or bash. For gdm3 (FSL9/10/11 and maybe later, the ~/.xsession file seems to interpreted using the account’s login shell. Thus for this case, source .login can be inserted as the first non-comment line to have it be used to set the environment. Comments were added to default versions to explain what to do.
3. Remove the -ls option for xterms in ~/.fvwm2rc and ~/.xsession files
  
  This option runs the shell used by the xterms as a login shell, i.e., it runs ~/.profile or ~/.login as appropriate when started. Since the environment was already set by gdm/gdm3, this was redundant.
4. Other ~/.fvwm2rc file cleanup.
  
  The entries in the Operator menu were column-aligned for easier reading.
  
  The Operator menu entry for monit7 was changed to Monit: DBBC3 to make it more specific than Monit: Tsys.
5. Bring ~/.fvwm2rc-before-FSL9 up-to-date
  
  The changes to the ~/.fvwm2rc files were also made to these files.
  
  Missing menu and hot-key items were added for: monit6, monit7, msg, rdbemsg, scnch, and erchk.
Update AUID default files for FSL11
Details
1. Add logic to AUID ~/.profile files to promote to oper/prog automatically
  
  At the end, the environment variable AUID_PROMOTE_ACCOUNT is checked. If it is set to oper or prog, then promotion occurs automatically (the AUID account password may still need to be entered). This variable is set by the FSL11 scripts oper_x11 and prog_x11, which then run startx to start X11 on the console. startx does not run ~/.profile (the environment from the AUID account is inherited). The /.xsession for the AUID account runs the xterm with the -ls option which triggers running ~/.profile. An exit command is used after the promotion script so that when the promoted session exists, the GUI is shutdown. This last step prevents the AUID account from being active on the GUI after the window manager has changed to the one for the promoted account.
2. Add default ~/.profile_<SUDO_USER> scripts to ~oper and ~prog
  
  A script is created for each AUID added. These scripts are run by the ~/.profile scripts for these accounts to allow per AUID user (the sudo user) customization. The default version of these scripts just prints a message as a reminder that it is being running. There is no need for the user to update the script file for their AUID account unless it would be helpful.
3. Cleanup ~/.fvwm2rc
  
  The operator menu was greatly simplified and renamed AUID menu. Access to FS program through this menu no longer makes sense. The AUID menu entries were column-aligned for easier reading.
Improve documentation
Details
1. Add appendix Transferring an existing FS installation to FSL11 to FS 10.2 Update Notes.
  
  This fills a gap in that there were instructions for how to update to FS 10.2 on an existing system and how to install FSL11, but there were no instructions for how to transition an existing system to FSL11. In principle, this should be part of installing FSL11, but it was much more cumbersome if included in the “FSL11 Installation” document. Instead, a pointer was added in that document, linking to the new appendix in the “FS 10.2 Update Notes” document. This may eventually be moved to the Installation Reference document.
2. Update Converting to a 64-bit System document.
  
  This document was revised to make it clearer how to use it for just transferring the files from an existing operational FS installation to a new system. Only a subset of the original steps is needed. Several other minor changes were made:
  
  Update for FSL11
  
  Use root account for transferring files if allowed. If not, use other appropriate accounts.
  
  User rsync to preserve ownerships.
  
  Add “new system” for logins to make it clear which system to work on.
  
  Reorganize to create reference copies of old system directories in a dedicated directory first, then make working copies as appropriate.
  
  Turn off write permission for reference copies.
  
  Recognize that <version> for /usr2/st-<version> may use other formats than semantic versioning.
  
  Turn off write permission for group and other for /usr2/st-<version>.
  
  Improve instructions for updating home directories.
  
  Copy old /etc, /usr/local/sbin, /usr/local/bin, and reference copies of older systems for reference on the new system.
  
  Fix some typos
3. Add a “Document revision history” section to some documents.
  
  Although the full revision history is contained in the git log listing, that can be complicated to interpret. The new section is intended to give an easier to read, very terse, synopsis of what has changed. Currently only the Converting to a 64-bit System and FS 10.2 Update Notes documents include this, but we expect to expand it to more documents.
4. Include updating calls to checkmk5 to checkfb in existing procedures, when appropriate.
  
  As part of changing to use procedure checkfb instead of checkmk5 for FlexBuff recorders for updating to 10.1, the user will need to change existing calls to the former to calls to the latter. This was omitted the FS 10.1.0 Update Notes update notes, but has been added now. A fix-up step for this has been added to the FS 10.2 Update Notes and FS 10.2-beta2 to FS 10.2 Update Notes This update is only needed if there are local procedures that use checkmk5. If both a Mark 5 and FlexBuff recorders are in use, separate versions of the calling procedures will be needed.
5. Add using ps2pdf to convert plotlog PostScript output to PDF.
  
  This may be useful for viewing the output on other systems that don’t support PostScript when giza is not being used.
6. Add missing monit7 continuous calibration changes to -beta1 update notes.
  
  Some of these were superseded by later changes. The changes for -beta1 include:
  
  Invalid T_sys with cyan background
  
  Negative T_sys with magenta background (superseded, now inverse)
  
  Change to N cal from Nccal
  
  monit7 changes were consolidated in a new item.
7. Improve the help output for the xresourcesfix script.
  
  Minor wording and format improvements were made. The help output is accessed by not providing any command-line arguments to the script.
8. Add opening a new window as an alternative to opening a new tab to avoid losing one’s place in a collapsible block.
  
  It may be easier than opening a new tab.
  
  This was added to the Links to different documents and to the inside of collapsible blocks subsection of the FS Font Conventions document and the Improve presentation FS change sub-item of the FS 10.1.0 Update Notes document.
9. Add using a link for a feature release to a pre-release document as an expedient.
  
  This will allow links in issues, discussions, emails, etc. that are created before the final release to still point to the latest information after the release. What is appropriate will need to be decided on a case-by-case basis.
  
  This was added to the Feature releases subsection of the Release Model document.

3.2. drudg changes

drudg opening message date is 2023-02-21.

None.