print to stderr from Matlab / GNU Octave

In both Matlab and GNU Octave, functions like error and warning by default print messages to stderr in non-interactive sessions, for easier capture to uncluttered log files.

To print to stderr in general, use

fprintf(2, 'my text')

Examples

You can verify these are printing to stderr by appending to the end of the command line command: 2>err.log

  • octave-cli --eval "fprintf(2,'this is stderr')"
  • matlab -batch "fprintf(2,'this is stderr')"

Windows native Meld diff

Git users often use Meld to resolve differences, three-way or two-way, graphically. Meld is also easily used with Git from Windows.

  1. Download/install Meld for Windows.
  2. Download/install Git for Windows.
  3. at Command Prompt type:

    git config --global diff.tool meld
    git config --global merge.tool meld
    
    git config --global mergetool.meld.path "%PROGRAMFILES(X86)%\Meld\meld.exe"
    

Windows with Cygwin or WSL git file permissions

When using native Windows Git client along with WSL or Cygwin Git clients, there can be nuisance dirty repo messages like:

old mode 100755
new mode 100644

This problem is easily resolved as follows, for each Git client you use as below:

  • Cygwin
  • Window Subsystem for Linux
  • native Windows
git config --global core.filemode false

Teaching the discovery process

The usual course instructor was unexpectedly out of town, and they thought I would be an ideal fit for their RF systems engineering lab course. I have a lifelong experience base in all aspects of RF systems design, so I thought I would turn this into a live learning opportunity for the students as well. I did precisely zero preparation for the course.

Upon arriving, I introduced myself as someone who had made a career in RF systems design before college. I said that I wanted to show them how I had to teach myself things as a formally educated engineering student, and we’d learn together the topic at hand. We used intelligently chosen web searches, some of which turned into dead ends. We looked at data sheets of commercial products to get a sense of what performance was reasonable for the RF subsystem we were designing. Then we set about to model the device, including S parameters and other KPI.

The point of this self-induced exercise was, we should not be intimidated by something we haven’t done before, or haven’t done recently. We shouldn’t be afraid to say I don’t know, in fact this is an asset of experience, to know the boundaries of one’s engineering competence. How I approach potential engineering contracts is that I say I will try to find someone with expertise in the needed area. Maybe it will be me, but if not I can help direct the prospective client to the needed expert.

That shows authentic confidence and competence in one’s practice area, and that one isn’t overreaching to get extra income that quarter.

Fortran build systems

We build all our Fortran projects using the CMake build system. We are increasingly using Meson as well as CMake as Fortran support in Meson is growing rapidly (due in part to our code contributions). The most important aspects of a build system are generation speed and build speed.

Generation speed

Regardless of team size, for non-trivial projects in any compiled language, build speed can be a significant pain point. Even for medium projects, the seconds lost repeatedly building add up to annoyance and lost productivity for developers.

In this regard, CMake and Meson are among the fastest generators, while SCons and Autotools are among the slowest.

Build speed

Ninja is known to be generally faster than GNU Make, particularly for the case where only a few out of very many files need to be rebuilt. This is the usual case for developers. So as long as the generator works with Ninja, you can get fast builds.

Meson generates Ninja build files. Meson 0.50 massively increased support for Fortran compilers and features, so check out Meson again if it didn’t work for you on Fortran previously.

using Meson vs. CMake

Meson is a Python ≥ 3.5 program without external dependencies. It’s easy to run the latest development snapshot (master Git branch) as a result.

git clone https://github.com/mesonbuild/meson
cd meson
pip install -e .

Since Meson generates Ninja build files, you will need to download Ninja. With Meson, the code build process goes like:

meson ..
ninja

while with CMake the code build is like:

cmake ..
cmake --build . -j

Modern Fortran submodule vs. preprocessor

Fortran has a long legacy of preprocessing systems, reaching back at least to the 1970s when C was not yet widely known. I would argue that modern Fortran ≥ 2008 has eliminated most preprocessors use cases. It’s usually much cleaner and quicker (build/runtime) to use Fortran submodule to switch functionality such as MKL vs. Atlas vs. Netlib Lapack using CMake, for example.

Numerous Python preprocessors have arisen (and sunk) along with the usual C/C++-based preprocessors. The only thing we use preprocessors for is very simple single-line statements, where adding a submodule would be significantly more cumbersome than a one-liner preprocessor statement.

Historically, very elaborate Fortran preprocessors were developed such that they nearly became a language unto themselves.

These extensive preprocessors have been made obsolete by advances from Fortran 77 onward.

CMake force linking static libs

CMake by default searches for shared libs before static libs. Sometimes a project needs to specifically link external or internal static libs, even if shared libs are present. A simple cross-platform way to do this is to include in CMakeLists.txt, before any find_library() or find_package():

if(NOT BUILD_SHARED_LIBS)
  set(CMAKE_FIND_LIBRARY_SUFFIXES .a .lib)
endif()

This takes advantage of de facto static library extensions for MacOS, Windows and Linux, using CMake system variables.

read huge RINEX files in Python with GeoRINEX

Reading RINEX files in Python or other languages historically required compiling or buying complex software. Several years ago, we created the GeoRINEX Python 3 program and library to be used with RINEX OBS and NAV files, including Hatanaka compressed files or other compression wrappers like ZIP.

Version 1.9 of GeoRINEX added the ability to read only every N seconds from a RINEX file with option georinex.load(..., interval=). This greatly speeds up reading where coarser time intervals than the RINEX file provides is needed. GeoRINEX uses performance-oriented techniques to read RINEX files at speeds approaching compiled languages, using Pure Python + Numpy and Xarray for metadata rich results.

If you use GNSS and RINEX in your work, you will probably like GeoRINEX for Python 3.

GUI programs to view HDF5/NetCDF4 data

Related: Check HDF5 files for corruption


Here are a few options for graphically browsing and editing HDF5 data files from Linux, Mac and Windows.

HDFview

HDFview is maintained by the curators of the HDF5 format, the HDF Group. HDFView enables editing (writing) as well as reading HDF5.

  • Linux: apt install hdfview
  • Mac, Windows download

HDFview needs compiling if running from source, unlike ImageJ, which is a pure platform-independent Java program.

Ubuntu 18.04

Ubuntu 18.04 (still!) has a broken hdfview such that the program opens, but silently fails to read any HDF5/NetCDF4 files. As an alternative, I use HDF-Compass on Ubuntu 18.04.

HDF-Compass

HDF Compass is also by the HDF Group. Unlike HDFView, HDF Compass is read-only.

apt install hdf-compass

Instead of using the menus, I typically open one or more HDF5/NetCDF4 files from Terminal like:

HDFCompass f1.h5 f2.nc

ViTables

ViTables is a Python-based HDF5 viewing program.

apt install libhdf5-dev

python -m pip install pyqt5

python -m pip install vitables

Unfortunately at this time PyQT5 doesn’t have a .whl for armhf, so that means ViTables only works on non-ARM systems (laptops, desktops, NOT Raspberry Pi).

PanoplyJ

PanoplyJ is downloaded for Mac, Linux or Windows. PanoplyJ does not require compiling, just run it using Java. Linux PanoplyJ requires Java 8 or newer, and can also work with Java 9.

apt install openjdk-8-jre

Extract PanoplyJ and make executable:

unzip PanoplyJ-4*.zip -d ~

chmod +x ~/PanoplyJ/panoply.sh

Run PanoplyJ by:

~/PanoplyJ/panoply.sh

Make a command-line shortcut by adding to ~/.bash_aliases

alias panoply='$HOME/PanoplyJ/panoply.sh'

If you get PanoplyJ error

Unknown superblock version=3

You may have an older version of PanoplyJ whose NetCDF-Java library isn’t ready for HDF5 1.10. If you’re writing the HDF5 files from h5py, you can use in your Python program writing the data:

with h5py.File('myfile.h5',libver='earliest') as f:
   ....

which should write in an HDF5 1.8-compatible way.

Install Gfortran 8.2 on Linux

Related: Install latest gfortran on Windows


Gfortran 8.2 Fortran 2018 features includes excellent coarray support.

Caveat: Newer versions of Gfortran may require recompiling other libraries linked with Fortran compiler, such as HDF5. Trivially switch between Gfortran compiler versions (or any other program versions) with update-alternatives.

Install

GCC/Gfortran 8 for Ubuntu 18.04 / 16.04 / 14.04 is available from the Ubuntu-test PPA.

Add Ubuntu-test PPA by:

add-apt-repository ppa:ubuntu-toolchain-r/test
apt update

Then install the most recent Gfortran (similarly for gcc-8, g++-8)

apt install gfortran-8

Switch between compiler versions with update-alternatives.

A popular alternative is Linuxbrew, which is officially part of Mac homebrew.

Notes

  • If you’re having errors getting any version of gfortran installed, try:
add-apt-repository universe
  • MacOS: install gfortran on Mac OS via homebrew:
brew install gcc