Document revision history

Click the Details toggle below to unhide/hide the history.

Details

  • 1.10.0 — Reorganize to put reference copies in their own directory first, them make working copies; use rsync instead of scp; minor cleanup

  • 1.9.1 — Add NOTE about not renaming /usr2/log

  • 1.9 — Remove extraneous log-old from chmod a-w …​

  • 1.8 — Improve customization of home directories; put Document revision history in a collapsible block

  • 1.7 — Note other formats for st-version

  • 1.6 — Transfer logs directly to a reference copy

  • 1.5 — Use root on the old system to copy files if possible

  • 1.4 — Add revision history

  • 1.3 — Rewrite to include FSL11 and make useful for more situations

  • 1.2 — Add using ssh-keygen

  • 1.1 — Cleanup

  • 1.0 — Add unlongify TIP

  • 0.8 — Fix typos

  • 0.7 — Font improvements

  • 0.6 — Simplify 64-bit upgrade path

  • 0.4 — Simplify 64-bit upgrade path

  • 0.3 — Initial version

1. Introduction

This document covers converting to a 64-bit system architecture from an existing FS 10.<x> installation on a 32-bit system. It is also used as part of the upgrade path to go directly to a 64-bit system while upgrading to FS 10.0.

The approach here can also be used to transfer an existing 64-bit (or 32-bit) FS installation to a new system without changing OS (including architecture) and FS version. For that case, in place of the Install a 64-bit system step make sure you have the same OS (including architecture) and FS version installed on your new system as on your existing system and then omit the Update station programs for 64-bit step.

If you are moving to a different OS, you may need to use a different FS version and update your station files and software for the new OS and FS version. For FSL11, a pointer to the instructions for this are included in its installation guide and in the Install a 64-bit system step below.

The process in this document assumes you will be installing to a new system. This will allow you to set up the new system without disturbing your operational system.

2. Convert to using a 64-bit system

Important
 All operations, except where noted, are performed on the new system.

Please follow the steps below.

2.1. Install a 64-bit system

Install a 64-bit (or 32-bit, but 64-bit is recommended) system on a different (hopefully new) computer. We recommend FSL11, which is based on Debian Bullseye and tuned for FS use. The FSL11 installation guide can be found at https://nvi-inc.github.io/fsl11/. That process installs FS 10.2. After that instead of the directions in the current document, you should follow the directions in the appendix Transferring an existing FS installation to FSL11 of the FS 10.2 Update Notes document. Those instructions include, by reference, a subset of the ones in the current document.

Note
 If you are following these directions as part of installing FS 10.0 on FSL10, that will still work, but we recommend using FS 10.2 on FSL11 instead.

You can install a non-FSLx Linux distribution. However, we can’t provide as much support in that case. The FSL11 distribution is tuned to provide a complete platform for running the FS. For other distributions you may have to make adjustments for several things that an FSL11 installation provides, including:

  • which packages are installed

  • required user accounts and groups

  • backup methods

2.2. Login as root on the new system

Login as root on the new system.

2.3. Rename default directories

Note
 This step assumes you have installed a base FS, either along with FSL11 or using your own methods. If the latter, and the normal default FS directories were not created, you can skip this step.

  1. Rename the existing default station specific FS related directories on the new system to get them out of the way:

    cd /usr2
mv control   control.DEFAULT
mv sched     sched.DEFAULT
mv proc      proc.DEFAULT

  2. If you use the /usr2/tle_files directory on your old system, you should rename the one on the new system:

    cd /usr2
mv tle_files tle_files.DEFAULT

  3. If your station software is in /usr2/st-0.0.0 on your old system, you should rename the one on the new system:

    cd /usr2
mv st-0.0.0  st-0.0.0.DEFAULT
Note
 The /usr2/log directory is not renamed since it will not be replaced with a copy of the directory from the old system.

2.4. Create directory for reference copies

It can be useful to have a reference copy of certain parts of the old system. In particular, you may want to preserve copies of the operational directories: control, log, proc, and sched; the last version of the station software; the home directories of oper and prog; old reference copies from previous updates; it may also be useful to preserve /etc. For this reason, the file transfer is organized around making reference copies and then making new working copies for items where that is appropriate. If you prefer, you can skip making reference copies and copy the operational directories of control, log, proc, and sched directly to /usr2 on the new system.

You can make a directory on /usr2 (usually the largest partition and one with ample room) to hold the reference copies. This might be, e.g., /usr2/old_FSL9, if your previous system was FSL9. You should use whatever naming scheme is appropriate for your situation. The current step and the following one use /usr2/old_FSL9 as an example. You should substitute the appropriate directory for your system.

You can create the reference directory structure with:

mkdir -p /usr2/old_FSL9/usr2

2.5. Transfer files

Your station specific FS related files can be transferred from the old system using any convenient method. If both systems are on the network this can be particularly easy. The following example sub-steps use this approach. You should replace the old host name in the examples with your old system’s host name. These commands must be executed as root, and except where noted, from the new system.

As an example, the sub-steps below use /usr2/old_FSL9 as the directory to hold the files from your system. You should substitute the appropriate directory for your system (see the previous step (Create directory for reference copies).

Tip
 If you want to copy-and-paste the example rsync commands below as is (this is possible for nearly all of them, the exceptions are the one for old:/usr2/st-version and possibly for old:/usr2/old_FSL8), you can add an alias old for your old system in the /etc/hosts file on the new system.
Tip

You may need to provide the appropriate password for each rsync command below. If so, you can simplify the process by copying your new system’s root public SSH keys to the old system with ssh-copy-id first. If root doesn’t have an ID yet, you will get an error like:

/usr/bin/ssh-copy-id: ERROR: No identities found

In that case, you can generate a key with ssh-keygen (leaving the passphrase empty is recommended) and try ssh-copy-id again.
Note
 By default -a option of rsync will copy symbolic links but not their referents (scp will follow the link and copy what it points to, the referent). This is usually what is appropriate for this application; it doesn’t pull files from other directories. If you want to copy the referents, please see the SYMBOLIC LINKS section of man rsync for more information (adding --copy-links will give the behavior of scp).
Caution
 It is assumed that the new system is on new hardware with disks more than large enough to accommodate all the data being transferred. However, if you installed the new Linux on old hardware with small disks, there may be a space issue.
Warning
 The instructions below assume that the old system you are using has a standard FS configuration in terms of symbolic links and directories. If your system is different, you will need to adjust what you do accordingly, but you may still find the outline of sub-steps useful.
Important
 These instructions assume that you can login to the old system as root using ssh. Allowing this is a security risk and should only be permitted temporarily for these command. If you are not able to allow it, you can use the oper and prog (or other) accounts on the old system to transfer the files. In that case, there is risk that some files may not be copied due to ownership/permission issues. You may need to take other steps to copy those files.

  1. Transfer operational directories:

    1. Transfer control, log, proc, and sched, from your old system:

      cd /usr2/old_FSL9/usr2
rsync -aH old:/usr2/control .
rsync -aH old:/usr2/log .
rsync -aH old:/usr2/proc .
rsync -aH old:/usr2/sched .
      Note
      		 Your log directory could be large and take a significant amount of time to transfer.

    2. Transfer tle_files:

      If you use the /usr2/tle_files directory on your old system, you can also transfer it:

      cd /usr2/old_FSL9/usr2
rsync -aH old:/usr2/tle_files .

  2. Make “starter” working copies of the operational directories (except log):

    cd /usr2
cp -a old_FSL9/usr2/control .
cp -a old_FSL9/usr2/proc .
cp -a old_FSL9/usr2/sched .

    and possibly:

    cd /usr2
cp -a old_FSL9/usr2/tle_files .

  3. Fix the ownership/permissions of the working copies you made:

    /usr2/fs/misc/fix_perm

    Answer y to confirm.

    Note
    		 If you don’t have a /usr2/tle_files directory, you will get a message that there is no such directory. That is benign unless you expect such a directory to be there.

  4. Transfer your station software directory (and make a working copy). This is usually the target directory pointed to by the /usr2/st symbolic link. On your old system, you can find its name with:

    ls -l /usr2/st

    In the rest of this sub-step, the target st-version will be used. You should replace version with your version string. For example, use 1.0.0, to make a target st-1.0.0. (You may need to adjust these commands if you have a different format for st-version, perhaps st-git.) If your target is st-0.0.0 you should rename the default on the new system first as described in the Rename default directories step above.

    All the sub-steps below are executed on the new system:

    1. Copy the target from the old system to the new system:

      cd /usr2/old_FSL9/usr2
rsync -aH old:/usr2/st-version .

    2. Create a local working copy

      cd /usr2
cp -a old_FSL9/usr2/st-version .

    3. Set the /usr2/st symbolic link to point to the target directory:

      cd /usr2
ln -fsn st-version st

    4. Set its ownership to prevent users other than prog from modifying it:

      cd /usr2
chown -R prog.rtx st-version
chmod -R go-w st-version

  5. Copy your oper and prog directories to the new system. This sub-step is optional. The FSL11 installation made default home directories for these users on /usr2. If you had not customized content for the users on the old system, you could just use the versions on the new system. Still it may be useful to have a copy of your old directories on the new system for reference, especially if you realize later that there were additional programs and files you want to use on the new system. You can accomplish the transfers as root using:

    cd /usr2/old_FSL9/usr2
rsync -aH old:/usr2/oper .
rsync -aH old:/usr2/prog .

  6. Transfer other files.

    You may need or want copies of other directories. These might include: /etc; /usr/local/sbin and /usr/local/bin (for RAID and AUID scripts); other directories from /usr2:

    cd /usr2/old_FSL9
rsync -aH --copy-links old:/etc .
rsync -aH --mkpath old:/usr/local/sbin usr/local/sbin
rsync -aH --mkpath old:/usr/local/bin usr/local/bin
...

    and possibly reference copies of even older installations (/usr2/old_FSL8 and/or others):

    cd /usr2/
rsync -aH old:/usr2/old_FSL8 .
...

  7. Set the permissions of the reference directory.

    You should probably set the entire directory to not have write permission:

    cd /usr2
chmod -R a-w old_FSL9

2.6. Optional cleanup

Important
 This optional step is to be performed by oper. Be sure to change accounts if you decide to do it.

If you would like, you can make a clean start by deleting old experiment files from /usr2/proc and /usr2/sched. Copies of the files deleted are preserved in the reference directory in case they are ever needed.

One possible strategy is to identify the most recent file you want to delete (the lt alias, or the ls -lt | less command, is useful for identifying it). In /usr2/proc, for example, that might be vo3347gs.prc. Working as oper enter:

Tip
 If you use this method, make sure that station.prc, point.prc, and any other procedure libraries that you use routinely, have been updated more recently than the file you identified. The touch command could be used to change their modification times. 
cd /usr2/proc
find . -type f ! -newer vo3347gs.prc | xargs rm -f

Likewise for /usr2/sched, the most recent file you want to delete might, for example, be vo3347gs.lst. Working as oper enter:

Tip
 If you use this method, make sure that any other procedure files (typically .snp files) that you use routinely, have been updated more recently than the file you identified. 
cd /usr2/sched
find . -type f ! -newer vo3347gs.lst | xargs rm -f

If you accidentally delete any files that you wanted to keep from either directory, you can copy them (as oper) from the reference directory when you discover they are missing. Files copied later in this scenario may require changes depending on the details of the FS updates you apply before then.

Other approaches can be used for delete files you don’t want. You could also delete any sub-directory structure in either or both of these directories, if you want.

2.7. Customize files on the new system

There are several things you may need to adjust on the new system. These include:

  1. Install additional Debian packages

    You may need to install packages for your new system that you had installed on the old system.

  2. Copy/merge additional system files, such as:

    • /etc/hosts

    • /etc/ntp.conf

    • /etc/hosts.allow

    • /etc/hosts.deny

  3. Customize the oper and prog home directories on the new system to include any features you want from the old system.

    This would typically include updating the default ~/.profile, ~/.bashrc, ~/.bash_aliases, and ~/bin. If you were previously using tcsh (the default for FSL9 and earlier) and are now changing to bash (the default for FSL10 and later), you will need to translate customized settings from your ~/.login and ~/.cshrc files.

  4. Update operational directories control, proc, and sched

    If you will be using a new FS version on the new system you may need to update files in these directories (and others). This is covered in the update notices you will need to apply for the new version.

  5. Update station software

    If you are moving to a new FS and FSLx version, any needed changes are covered in the FS update notice. If you are converting to 64-bit, instructions can be found in the following steps.

    If there is other software you use, you would need to install that as well.

2.8. Login as prog on the new system

The following steps up to Reboot should be executed as prog on the new system.

2.9. Update station programs for 64-bit

This step is for modifying your station programs in /usr2/st. There are two possible issues, conversion of FORTRAN code and conversion of C code.

Note
 If you are not converting from 32- to 64-bit, you should skip this step and go directly to the Make local software step below.

2.9.1. Conversion of FORTRAN code

If you don’t have any FORTRAN station code or you have already converted it to f95, skip this sub-step.

Use of f95 is necessary on 64-bit systems. If you have station programs in FORTRAN, please email Ed so he is aware.

You will need to adapt your Makefiles to use the same compiler options as the FS, which can be found in /usr2/fs/include.mk. As a first cut, it may work to add the following two lines to your Makefiles for FORTRAN programs:

FFLAGS  += -ff2c -I../../fs/include -fno-range-check -finit-local-zero -fno-automatic -fbackslash
FLIBS   += -lgfortran -lm

2.9.2. Conversion of C code

If you don’t have any C station code, you can skip this sub-step.

If you have C station code, it should work as written unless you have declared integers that interface to the FS as long (this was actually a common practice due to the example code that been provided). For a start at fixing those, please see https://github.com/dehorsley/unlongify. The following sub-steps describe how to install and use the unlongify tool.

2.9.2.1. Install go language

If you haven’t already, you will need to first install the go language. If you are using FSL11, you can install the go language in one of two ways listed below:

  1. Option A - Installing golang package

  2. Option B - Installing latest go language

We recommend the first way for those that are only using go for the unlongify tool. After installing the go language, continue the Conversion of C code instructions starting at Configure prog account for go language sub-step below.

Option A - Installing golang package

You can use the Debian package management system to install go. This will give you an older version of go that is perfectly adequate for the task at hand and is supported by the normal security update mechanism. To install it this way, as root use:

apt-get install golang
Option B - Installing latest go language

You can install the latest version of go, but this is outside the normal security update mechanism. In this case, you will need to manage your own updates, which may not be suitable for an operational environment. If you use this method it is recommended that you sign-up for go language announcements so that you will be informed when a security update is available. You can sign-up at https://groups.google.com/forum/#!forum/golang-announce.

Another alternative is to delete the latest go (rm -rf /usr/local/go) after you have made unlongify. You can always re-install it if you need it again.

Both the initial install and updates are handled by the fsadapt script, as root:

cd /root/fsl11
./fsadapt

In the first window select only the option (i.e., only that line has a *):

goinst    Install (or 'Update') Go programming language

Then press Enter while OK is highlighted. On the next screen, press Tab to highlight Cancel and then press Enter.

2.9.2.2. Configure prog account for go language

Once you have the go language installed, you need to define the GOPATH environment variable and include it in prog's path. The default ~prog/.profile file includes two commands (commented out by default) to accomplish these things:

#export GOPATH=~/go
#PATH="$GOPATH/bin:/usr/local/bin/go:$PATH"

You will need to uncomment these two lines and then logout and log back in again as prog or, in a current login session for prog, re-execute the file:

. ~/.profile
2.9.2.3. Install unlongify

Then you should be able to execute the installation step given at the URL above (as prog):

cd
go get github.com/dehorsley/unlongify
2.9.2.4. Use unlongify

Please read the README.md file, which is displayed at the URL above. Alternatively, it can be viewed at ~/prog/go/src/github.com/dehorsley/unlongify/README.md where it was installed by the above command. Please pay particular attention to the Note about system calls.

Tip
 The unlongify program attempts to process all *.h and *.c files in the path specified as its argument. If it encounters a file with a syntax error, an error message will be printed and processing will stop. Presumably, this would not happen for files that are in active use, e.g., are referred to in Makefiles. However, there may be code with issues in files that are not currently used. If an error occurs, the messages should help you identify files with issues, which can moved (re)moved or corrected as appropriate. Thanks to Carlo Migoni (Sardinia) for reporting this and spurring the addition of file names to the error messages.

2.10. Make local software

Important
 If you are installing a 64-bit system as part of upgrading to FS 10.0, you should return to the FS update instructions at this point.

If /usr2/st/Makefile is set-up in the standard way, you can do this with:

cd /usr2/st
make rmdoto rmexe all

Once your code makes successfully, you can continue, but you may need to debug it in the Test the FS step below.

2.11. Reboot

Important
 Reboot the computer. This is important for initializing shared memory for the new version.

2.12. Login as oper on the new system

The remaining steps assume you are logged in as oper on the new system.

2.13. Test the FS

Note
 You may need to debug your station code as part of this.