FS 10.1.0 Update Notes

The operator output of onoff was completely restructured. The raw data is still recorded in the log, but now the operator will see only summary messages that describe what the program is doing and the final results (the VAL lines). Each major action that the program undertakes is reported, as well as any steps needed to recover if an error should occur. This should make the output both more compact and useful. The full output can still be seen in the Log Display window by using xdisp=on (use xdisp=off afterwards to turn off the more verbose display).

Previously, for systems that require locking the gains, they were set before confirming that the antenna had reached the starting position. Now onoff waits until it has been reached. This will have no impact when onoff is launched already at the starting position, including when run by aquir.

If onoff is started before the antenna has reached the starting position, the effects will be:

Previously, these errors were ignored due to a bug. Now onoff will detect antcn errors. If one occurs, onoff will repeat the request to antcn, one time, to try to make itself more robust if a transitory antenna communication problem occurs.

It seems unlikely that these changes will cause a problem, but if they do, a short-term workaround is provided. If the environment variable FS_ONOFF_SUPPRESS_ANTCN_ERRORS is set to 1, the previous behavior will be restored. In the longer term, it may be possible to change your antcn to avoid this issue.

The default for pushing files (logs) to the BKG data center is now ftp-ssl. Unfortunately, accessing BKG with ftp-ssl cannot be supported on FSL8. The old behavior using non-SSL ftp can be restored by setting the environment variable PLOG_BKG_METHOD to ftp. However, BKG is expecting to discontinue support for non-SSL ftp after June 9, 2022.

OPAR now uses a different URL, https://ivsopar.obspm.fr/upload/, for uploading. The old one, https://ivsopar.obspm.fr/upload/scripts/upload.php, will continue to work for some time.

Thanks to Sébastien Lambert (OPAR) for sending the new URL, and Christophe Barache (OPAR) for keeping the old one open for backward compatibility for at least some period.

plog will now use bzip2 as the default program for compressing _full.log files to send to the data centers. It is possible to use gzip instead by setting the environment variable PLOG_COMPRESSED_EXT to gz or using the -g command line option. Please use plog -h for more information.

In the process of reducing a log (removing multicast data), the log is renamed. This can cause a log recovery to occur if the log is currently open in the FS since it appears to be missing. This was protected against by plog, which would refuse to rename the log if it was open to any program. This meant that if the log was, for example, being viewed with tail -f or less it could not be renamed. This was fixed so that plog will only refuse to rename the log if it is open in the FS (to ddout)..

Thanks to Katie Pazamickis and Jay Redmond (both at GGAO) for reporting this.

Before, when a log contained multicast data, the default was to only upload a reduced log (with multicast removed). Now both the reduced and compressed full log are uploaded by default.

A new option, -r, was added for uploading just a reduced log. As before, the -z option will upload just a compressed full log. It is not expected that either of these options will get much use. The -r and -z options cannot be used together.

Thanks to Chevo Terrazas (MGO) for suggesting uploading both file by default.

This case had been overlooked in closing #39, which had added the NETRC_DIR environment variable.

Thanks to Kiah Imai (KPGO) for reporting this and testing the fix.

This change was made to improve ease-of-use when specifying the data center with the -c option. For consistency, the value specified by the DATA_CENTERS environment variable is now also case insensitive.

This change makes the use of the STATION environment variable by plog and fesh (the latter now also case insensitive), consistent.

The default for pulling schedule (and .txt note) files from the BKG data center is now ftp-ssl (using anonymous access). Unfortunately, accessing BKG with ftp-ssl cannot be supported on FSL8. The old behavior using non-SSL ftp can be restored by setting the environment variable FESH_BKG_METHOD to ftp. However, BKG is expecting to discontinue support for non-SSL ftp after June 9, 2022.

With -P, when drudg is run by fesh, it will print the summary directly to the printer. It is appropriate to make this a fesh option since it is an integrated feature of drudg. To print other files, it is recommended to make a wrapper for fesh. An example wrapper, which includes the normal processing, prints the summary, and prints the .prc file, is included as /usr2/fs/fesh/feshp. This can be copied to ~oper/bin and possibly customized to print different files.

Thanks to Kiah Imai (KPGO) for suggesting this.

This allows fesh to trigger the normal drudg processing when the schedule is already on the disk. This might be useful for example, if the schedule was generated locally by shifting the schedule (drudg option 10).

Support for new environment variables FESH_GEO_USE_SETUP_PROC and FESH_GEO_VDIF_SINGLE_THREAD_PER_FILE were added. These variables can be used to supply fixed answers to the new optional drudg prompts as described in drudg changes below. For more details about the use of the new variables, please see the fesh -h output.

This change was made to improve ease-of-use when specifying the data center with the -D option. For consistency, the value specified by the FESH_DATA_CENTER environment variable is now also case insensitive.

Before the station code was expected to be lower case. If it wasn’t, this could cause a conflict with drudg, which maps it to lower case. This could result in a mismatch on the file names for deleting .snp and .prc files with the -f option. fesh maps the station code, from both the STATION environment variable and -s option, to lower case.

Thanks to Eskil Varenius (Onsala), for suggesting this.

Two command line options were added to rdbemsg:

This was added to allow distinguishing the early version of MCI node at Westford, which requires different handling. Specifically, the MCI logs are in the directory ~oper/node_software/V0, the file names do not contain the station code, the fields in the file are space delimited, and the fields are in a different order. The correct form for Westford is:

Other stations do not need this parameter and can either not include it or comment it out.

The example control file, /usr2/fs/st.default/control/rdbemsg.ctl, was updated accordingly.

Previously, if you zoomed in the T_cal(K) versus frequency plot and selected display of the working file T_cal(K) curve, it would reset the left plot edge to the un-zoomed value when drawing the curve. This was caused by an interaction of two issues:

This was fixed by limiting the T_cal(K) curve plotted to just segments within the zoomed area. In addition for consistency, the drawValues function was change to reset any plot limit that is exceeded by the data. Since all other uses of drawValues already limited the horizontal values to be plotted to the zoomed area, there was no impact on those other uses.

Thanks to Eskil Varenius (Onsala), for reporting this issue and testing the fix.

This was caused by an incorrectly sized array in gndat that was introduced in commit f84a2bb9 dated June 2003. This error was benign unless more than 20 detectors were used in a single onoff run. Before the advent of RDBEs and DBBC3s, this was unlikely, but not impossible.

Due to the layout of the automatic variables, this error should only have impacted \(\mathit{T_{sys}}\) and \(\mathit{T_{sys}-T_{spill}}\) data in gnplt and only when there were more than 20 detectors used. However, also considering the layout of the automatic variables, it is unclear why there were not catastrophic program failures in such a case. It would be prudent to reanalyze any current data sets that used more than 20 detectors with the fixed version to see if the results change.

Thanks to Beppe Maccaferri (Medicina), and subsequently Eskil Varenius (Onsala), for reporting this issue.

The debug output was not visible when gndat was run from gnplt, which is normally the only way it is run.

This change is intended to make it apparent that the calculation used in old refrw() routine is incorrect and encourage use of the correct refrwn() routine. The error in the old refrw() routine has been known about since 2006. The error is generally small above 5° elevation.

The old routine can still be used by changing calls to use its new name. This may be preferable in the short-term if the effect of this error is built into current point models. To get consistent results, station code can be converted to use refrwn() and a new model determined, when it is convenient to do both.

Thanks to Jon Quick (HartRAO) for pointing out this error and supplying the corrected refrwn() in 2006.

Two additional refraction calculations are now available in the C language poclb.a library:

The Lanyi algorithm is expected to be the best of the ones in poclb.a.

This should not be a significant effect above 5° elevation. Refraction is only applied to estimate an approximate value of the antenna elevation; great accuracy is not required. In any event, fixed values are used for the meteorological parameters, which limits the accuracy. For fivept, the FORTRAN refr() function is used, which also has fixed meteorological parameters.

The older refrwn() (C) and refr() (FORTRAN) routines are apparently derived from W. R. Iliff and J. M. Holt, “Use of Surface Refractivity in the Empirical Prediction of Total Atmosphere Refractions,” J. Research NBS 67D (Radio Prop.), No. 1, pp. 31-35, January-February, 1963 (https://www.semanticscholar.org/paper/Use-of-surface-refractivity-in-the-empirical-of-Iliff-Holt/1ddcd0e4e672dd890198539361c5237c033001f7) and C. A. Clark, “Haystack Pointing System: Radar Coordinate Correction,” MIT LL Technical Note 1966-56, 24 October 1966 (https://www.semanticscholar.org/paper/HAYSTACK-POINTING-SYSTEM%3A-RADAR-COORDINATE-Clark/bf564e4ebc49a4ae8d69333b267a97cc320109a0).

Thanks to Ludwig Schwardt (SKA, South Africa) for tracking down these references.

See the README.txt file in the directory for more information.

This uses the same method as the FS source=…​ command.

See the README.txt file in the directory for more information,

This program has a set of comparisons hard-coded, but could be modified to test other combinations.

As of 10.0.0-beta1, the previous practice of disabling receipt of certain signals, in particular SIGINT (for Control+C), had been removed. It has been reinstated. When the FS is run without the display server, this assures that no FS programs will be aborted if a Control+C is accidentally entered in the terminal session where the FS is running. However, this does not prevent a Control+C from killing an xterm that is wrapped around a FS program (typically only oprin) in the FS terminal session from being killed. Preventing that is discussed next.

To prevent Control+C killing an xterm in the FS terminal session all x programs in fspgm.ctl (just oprin) and stpgm.ctl are now run under setsid() when the display server is not being used. This disconnects the programs from the terminal session that the FS is being run in, thereby preventing a Control+C from reaching them, but not otherwise affecting them.

The error messages when reading .rxg files were confusing and uninformative. This has been the situation since .rxg files were first added in October 2002, for version 9.5.15 (commit 1b68b90f). Apparently, this was not a significant issue because .rxg files were usually updated by gnplt, which wrote correctly formatted lines. If you have been having trouble with this, we apologize.

The error messages have been more explicit about the cause of any read error. Unfortunately, it is not possible to show the offending line without more significant changes. However, the messages are fairly explicit about where the error occurred.

The same reading routine is used in gnplt (which uses gndat to read the .rxg files) for consistency. Unfortunately, the error messages for gnplt will still not be as informative, but restarting the FS should provide a more explicit error message that help make it clear what the problem is. If it is not possible to use the FS for this, another strategy is to use the error number reported in status line at the bottom of gnplt to identify the corresponding RG error in /usr2/fs/control/fserr.ctl.

While it is unlikely, if any of your .rxg files have these errors, they will be reported the first time you run the FS after the update. You can correct them at that time. The error messages should identify the problem clearly enough.

A few other, minor, errors are still not being caught. In particular, if a character that cannot be part of a numeric field appears within, or at the end of, a numeric field, that error will not be reported. In that case, the field up to the out-of-place character will be used as the value. Hopefully this is an unlikely error. A leading character in a numeric field that is inconsistent with the field being a numeric value will be reported as an error.

These conventions are covered in the FS Document Font Conventions document. The following descriptions refer to sections in that document. The conventions themselves are covered in the Conventions subsection. Examples are in the Source examples section.

Using differently named ~/.Xresources files that are used by different aliases for different displays is described in the final NOTE of the Setting geometry values in .Xresources section of the Installation Reference document.

This is technically more accurate. The change primarily affects the (now) Copy-and-paste installation tips in the Installation Reference document and references to it.

In the update notes for FS 10.1.0 (this document) the list of FS and drudg changes now typically show a numbered title, a summary paragraph, and a Details toggle to show (or hide) a more detailed description. This should make the list of changes easier to review while allowing a reader to view more details for items they find interesting. We hope this will make the update notes easier to read.

There are some downsides, primarily:

If there is a consensus that this is overall an improvement, its use will continue for future updates.

Section Setting environment variables was added to the FS environment variables document.

The preamble of the Runtime variables section in the same document was expanded to include the additional steps needed to update environment variables when using the display server.

An example check_ntp procedure is provided in check_ntp.prc. It is slightly improved from the original version. It has also been incorporated into the example station libraries for the current most commonly used rack/recorder combinations: DBBC/Mark5B, DBBC/FlexBuff, DBBC/March 5C, DBBC3/Flexbuf, and VLBA5/Mark 5B. Please see the table in the Example standard procedure libraries section of the Installation Reference document to identify which files are for which combinations.

Before the current release there was no way to know which BBCs or IFs were being recorded to implement these groups. Now that the core3h_mode command is implemented, it is possible.

For T_cal(K), this generated incorrect values, which impacted T_sys values by a proportional amount. The incorrect values were for a frequency in the band or nearby, so typically not entirely unreasonable.

Since T_sys calculated by the FS is only a station diagnostic, this had limited impact. Before the current release, T_sys for DBBC3’s, could effectively only be calculated in fivept and onoff and to spot check T_sys outside of schedule execution. Pointing offsets (from fivept) and SEFDs/T_cal(J)s (from onoff) were not affected. The T_cal(K) values were probably adequate for verifying consistency of station performance with T_sys spot checks, if they were used at all.

For frequency dependent FWHM, this should have been an insignificant error in most cases.

The mout6() function was using an int to hold the time instead of a time_t, resulting in incorrect values for the nominal VDIF epoch on 64-bit systems. This had very limited impact. The nominal VDIF epoch was only used to apply inverse video to epochs that weren’t nominal when all RDBEs had different epochs, a rather unlikely case. If it occurred, all the VDIF epochs would have been shown in inverse video as “wrong,” which would have been “incorrect” for at most one epoch. This has been fixed.

The error was introduced by the bulk unlongify for conversion to the 32/64-bit compatibility for version 10.0.0. It only would have impacted 64-bit systems.

The VDIF epoch increment command (>) hint line, which only appears if the RDBE’s VDIF is before the current epoch, was displayed twice. Additionally, if it became no longer useful, it was not removed. In the version 9.12.13, the hint lines dynamically adjusted depending on whether an increment was possible. This behavior was not correctly preserved in the merge of versions 9.13.2 (old “main”) branch and 9.12.13 (old “vgos” branch). The new version uses statically placed hints lines for simplicity, only showing the VDIF epoch increment line when it could be useful.

Previously, it was possible to set the 5B/Ethernet sample rate via the decimation parameter in the mk5c_mode (and fb_mode) command. This was a vestige of the mk5b_mode command. The Ethernet recorders (Mark 5C and FlexBuff, controlled by jive5ab) do not support decimation. The sample parameter of the mk5c_mode (and fb_mode) command provides full functionally for specifying the sample rate for 5B/Ethernet and VDIF recording and is the method used by drudg.

None of the devices these commands query report the sample rate. For fila10g_mode and mk5b_mode, the sample rate implied by the device decimation response is shown in parentheses. Previously, they reported the commanded rate in parentheses.

For mk5c_mode/fb_mode, the devices also don’t report the decimation, so the commanded sample rate is shown in double parentheses. The device does report the track bit rate, which can be used for comparison, but this is reported in bits per second instead of Ms/s.

This was probably benign since it was unlikely that two channels represented by the top four mask bits were the only ones with two-bit sampling. This would only have affected systems with a FiLa10G.

In FS 9.7.0, July 2004, commit acde80ba2 the value if π used by skdrut/prefr.f was updated with a value with more digits. The current change catches fslb/pref.f up with that previous change.

This increased digits changes the value of π by somewhat less than 3 parts in 10^-11, which should be insignificant. This marginally improves the consistency of FS precession rotations between B1950 and J2000 with those in drudg.

Some error messages were displayed/logged with extraneous (xx) strings after the error number. This came about because a change was made (in commit 9e9906d9 for version 9.12.2 released in March of 2016), to report the so-called LWHAT two character Hollerith values (or up to four character representation of numbers) for an error even if there was no ?W (or ?WWW) string in the message to be replaced with the value. The principle being that it was more important to report the information to help solve a problem than to adhere to the possibly incorrect formatting of the error message.

This did not seem to cause much of a problem. However, some programs, in particular aquir, fivpt, onoff, were incorrectly setting the LWHAT value. Other programs, in particular mk5cn, dbbcn, and rdbcn, were carelessly setting the LWHAT value when there was no error. The latter problem caused extraneous strings in error messages from unrelated commands. We believe all cases of this have been fixed. If you see any now, please report them.

Previously, if an error message had a runtime determined numeric value, there had to be a four character ?WWWW placeholder in the error message for it to replace to avoid overwriting. This was changed to allow shorter placeholders, down to ?W, to be used. A ?W can be used to show a one digit number with one space in front of it. To do this, the ?W must be directly appended to the previous word in the message, for example word?W, which may be a little hard to read in the fserr.ctl, or sterr.ctl, file.

Previously, the message for reporting errors when sending data to the Ethernet devices (Mark 5s, DBBCs, RDBEs, Mark 6s), incorrectly stated that the connection was closed. The connection is not closed. Perhaps it should be, but the incorrect statements have been removed until the functionality is changed. The affected errors are: DB/M5/M6/RA -102.

This error is reported by setcl when the formatter and FS PC boot time differ by more than 248 days. This may occur because the PC has been running a long time since it was booted, but it also could be because the formatter time has not been set or is wrong. The error message now explicitly suggests checking both.

If an error occurred in the monitor form of the command, the error was reported as di instead of dg.

Instructions for implementing this were added to the FS 10.0.0 Update Notes document as the Updating ~/.xsession sub-step in the Miscellaneous FSLx changes step.

The .fvwm2rc files in the auid, oper, and prog sub-directories of /usr2/fs/st.default were updated. These changes only affect behavior on the console GUI.

An argument was missing. This was a bug from the VGOS branch, which technically do not support using setcl for FiLa10G. The bug apparently did not affect versions since the merge since there was a relatively low, not quite 1 in 2³² chance of it being excited.

Currently, this parameter is used in the syntax for the device, but is meaningless.

Before the current release, there was no way to know what channels were being recorded. Additionally, direct sampling was too slow to be useful. Only recorded channels would be sampled now, but direct sampling is no longer supported in favor of multicast.

This is the normal structure of SNAP commands, which mk5_status did not have. It enables only updating control/fscmd.ctl to either change the name or create a synonym. Fixing this was necessary in order to make the fb_status synonym work correctly.

help for the pcald and tpicd commands now works for all systems.

If enabled and if the target_correlator parameter in a VEX schedule includes the string e-vlbi (case insensitive), drudg will:

The equipment selection can still be changed subsequently with drudg option 11.

If enabled, drudg will detect gaps in the schedule at least as long as a user settable max_gap_time, say an hour, and wait to start the next scan a user settable pre_time time before it is scheduled. The pre_time should probably be short, but at least as long as the worst case slewing time of the antenna. The concept is that the scan at the start of gap is closed, i.e., check…​ procedure is run before slewing to the next source. Then there is a wait until the pre_time before the next scan will begin for the scan_name=…​ and source=…​ to be commanded.

This can be useful for intensive schedules. The antenna won’t slew needlessly to a new source or setup for new scan at the start of the gap. The schedule can be broken into while it is waiting for the next scan to start, the intensive schedule run, and then the original schedule rejoined for the next scan.

If enabled, drudg prefixes each setup procedure call with setup_proc=. For more information see the Minimizing the use of setup procedures appendix of the FS DBBC3 Operations Manual.

This option allows control of whether the VDIF files created by Mark 5C and FlexBuff recorders are multi-threaded or single threaded.

For more information see the Thread procedure appendix of the FS DBBC3 Operations Manual.

These bugs apparently first appeared in version 10.0.0.

Defaults for up to eight IF inputs can be specified for DBBC3s.

Values for up to eight IF inputs can be specified for DBBC3s.