Using Git directly with Overleaf

To avoid having to constantly type your username for https://git.overleaf.com (or any other Git HTTPS connection), set in your main ~/.gitconfig file:

[credential "https://git.overleaf.com"]
    username = your@email.address

Use the same email address you use to login to Overleaf. You will still need to type your password

Optionally, you can set a Git credential cache. The command below remembers your password for 300 seconds (5 minutes).

git config --global credential.helper 'cache --timeout=300'

The Git credential cache is for HTTPS Git credentials, SSH credential caching is not affected.

f2py on Windows including AppVeyor CI

Related: AppVeyor Windows/Linux CI templates for Python and Fortran


Importing Fortran libraries from Python is easy and highly effective. We write modern Fortran code from scratch explicitly for the purpose of importing into Python. Here is an example .appveyor.yml script useful for CI with f2py.

image:
- Visual Studio 2017
- ubuntu1804

stack: python 3

environment:
  PY_DIR: C:\Python37-x64
  MINGW_DIR: C:\mingw-w64\x86_64-7.2.0-posix-seh-rt_v5-rev1\mingw64\bin

clone_depth: 25

build: off

init:
- cmd: set PATH=%PY_DIR%;%PY_DIR%\Scripts;%PATH%
- cmd: set PATH=%MINGW_DIR%;%PATH%
- ps: if ($isWindows) {echo "[build]`ncompiler=mingw32" | Out-File -Encoding ASCII ~/pydistutils.cfg}

install: pip install -e .[tests]

test_script: pytest -rsv

This pyproject.toml ensures Numpy is installed before attempting to build the Fortran extension module:

[build-system]
requires = ["setuptools", "numpy"]

setup.py must import setuptools, even though it isn’t directly used here, to enable developer mode via pip install -e .

import setuptools  # noqa: F401
from numpy.distutils.core import setup, Extension


setup(ext_modules=[Extension(name='mymod', sources=['mylib.f90'])])

f2py import Fortran code in Python

Related: simple F2py Fortran examples


Importing Fortran code in Python just like any other Python module is very straightforward, using F2py.

Prereqs

On any operating system, a Fortran compiler and Numpy are required to use F2py. If you don’t already have a Fortran compiler, we suggest GNU Gfortran.

  • MacOS / Linux: using Homebrew brew install gcc
  • Linux apt install gfortran or similar

Windows

Windows requires a one-time system-wide configuration to use F2py properly. Tell Python to use MinGW by creating file %USERPROFILE%\pydistutils.cfg containing:

[build]
compiler=mingw32

Do this from Powershell by copy/paste this line:

echo "[build]`ncompiler=mingw32" | Out-File -Encoding ASCII ~/pydistutils.cfg
  1. install MinGW-W64 to c:\mingw with:

    ArchitectureThreadsException
    x86_64posixseh
  2. Insert the mingw bin path to your system path permanently: Edit Environment Variables for Your Account → User variables → Path. Add the path to gfortran.exe, such as:

    C:\mingw\mingw64\bin
    

You will need to manually find that directory one-time on your system. Open a new command prompt and be sure gfortran is found.

Test/fix

Try the lowtran7 code. Following the instructions there, you should get

  1. A lowtran7.cp37-win_amd64.pyd (on Windows) or lowtran7*.so (on Mac/Linux) file
  2. Running python DemoLowtran.py creates a plot of atmospheric loss

Lowtran output

f2py does not allow inline comments for COMMON blocks

f2py does not allow inline comments for COMMON blocks for Fortran 77 .f code. This is because f2py works more strictly to Fortran specifications than most modern compilers.

Inline comments are not Fortran 77 standard, and will make f2py throw an error.

To fix this problem, just make the inline comment a full-line command with ! in column 1.

Fortran90 .f90 files won’t throw an f2py error due to inline comments on a line with a COMMON block: goodcomment.f90.

This will manifest itself two different ways, depending on whether you have implicit none or not:

COMMON inline comment error WITH implicit none

Example in badcomment_implicit.f

var2fixfortran: No typespec for argument “x ! bad for fortran77”. getctype: No C-type found in “{}”, assuming void. KeyError: ‘void’

Solution: Make inline comment a full-line comment with ! in column 1.

COMMON inline comment error WITHOUT implicit none

Example in badcomment.f

error: expected ‘;’, ‘,’ or ‘)’ before ‘!’ token

Solution: Make inline comment a full-line comment with ! in column 1: goodcomment.f.

Windows troubleshooting

Another solution is to use Windows Subsystem for Linux with Anaconda Python. However, with the techniques below, I’ve always gotten f2py to work on Windows 7 and Windows 10.


‘f2py’ is not recognized as an internal or external command, operable program or batch file.

Ensure Numpy is installed

conda install numpy

(Windows) create a file <Anaconda Python install directory>/Scripts/f2py.bat with content

python %~dp0/f2py.py %*

Error: No module named ‘numpy.distutils._msvccompiler’ in numpy.distutils; trying from distutils

is fixed by: create file %USERPROFILE%\pydistutils.cfg containing:

[build]
compiler=mingw32

error LNK2001: unresolved external symbol _gfortran_st_write error LNK2001: unresolved external symbol _gfortran_st_read

and similar errors are typically from not having told f2py to use gfortran by: create file %USERPROFILE%\pydistutils.cfg containing:

[build]
compiler=mingw32

If you don’t want to create %userprofile%\pydistutils.cfg as recommended above, you can instead do for each F2py package you install:

python setup.py build_ext --inplace --compiler=mingw32

If you have problems using f2py or other development work on Windows, consider Windows Subsystem for Linux, which runs at full performance within a terminal window on Windows.

Notes

Reference

One-step build/install CMake

Because of the rapidly growing CMake feature set, Fortran users are encouraged to use the latest CMake version.

Compile/Install CMake

This will get you the latest release of CMake. For Linux and Mac, admin/sudo is NOT required.

CMake major versions

  • 3.15: CMAKE_GENERATOR environment variable works like -G option
  • 3.14: check_fortran_source_runs()
  • 3.13: ctest --progress, better Matlab compiler support, lots of new linking options, fixes to Fortran submodule bugs
  • 3.12: transitive library specification (out of same directory), full Fortran Submodule support
  • 3.11: specify targets initially w/o sources
  • 3.10: added Fortran Flang (LLVM) compiler, extensive MPI features added
  • 3.9: further C# and Cuda support originally added in CMake 3.8.
  • 3.8: Initial Cuda support
  • 3.7: comparing ≤ ≥ , initial Fortran submodule support
  • 3.6: better OpenBLAS support
  • 3.5: Enhanced FindBoost target with auto Boost prereqs
  • 3.4: Limit CPU usage when using ctest -j parallel tests
  • 3.3: List operations such as IN_LIST

Thoughts on Overleaf v2, Git and GitHub

I am happy with the upgrades available for Overleaf v2. My main reservation is the GitHub permissions coarseness described below. Instead of GitHub, I use the direct Git Overleaf v2 interface that was released for Overleaf v2 in January 2019.

GitHub permissions

Overleaf v2 asks for read/write permission to ALL public & private GitHub repos.

This broad permission ask by Overleaf v2 is not acceptable, unless I created a separate GitHub account just for Overleaf. It’s due to Overleaf v2 being based on ShareLaTeX code using the legacy GitHub API.

Overleaf should update to GitHub Apps platform, which allows granular per-repository permissions. The Travis-CI switch to GitHub Apps had permissions granularity as a key motivator.

Open source

Parts of the Overleaf v2 code base are open source, due to ShareLaTeX being open source, which Overleaf v2 is based on.

Offline Overleaf

While Overleaf GUI doesn’t work offline, you can edit Overleaf v2 LaTeX documents offline from a Dropbox folder or Git repo on your computer. Offline Overleaf v1 was also possible, but considerably more cumbersome than with Overleaf v2.

Install MSYS2 on Windows

MinGW has been bringing GNU compiler tools to Windows since the late 1990s. MSYS2 provides a superset of tools including MinGW, with more complete and up to date GNU compiler set, using pacman package manager.

Install

  1. download msys2-x86_64-*.exe
  2. run the installer, installing to c:/msys64. It needs to be on a non-FAT/ExFAT drive, such as c:.
  3. To run, start the MSYS2 console in the Windows Start menu, which sets up all the needed paths, just for that terminal session.
  4. do an update to get the latest packages, in the MSYS2 terminal. Run this command multiple times until it says “nothing to do”.
   pacman -Syuu

To use MSYS2 / MinGW64 programs from regular command prompts, add MinGW64 to Windows PATH, typically C:/msys64/mingw64/bin

Usage

From MSYS2 command prompt, tasks include:

Search for packages:

pacman -Ss gcc

Install MinGW packages

  • gcc: pacman -S mingw64/mingw-w64-x86_64-gcc
  • gfortran: pacman -S mingw64/mingw-w64-x86_64-gcc-fortran
  • make: pacman -S mingw64/mingw-w64-x86_64-make
  • pkg-config: pacman -S mingw64/mingw-w64-x86_64-pkg-config
  • aspell: pacman -S mingw64/mingw-w64-x86_64-aspell mingw64/mingw-w64-x86_64-aspell-en

If you have confusion about why another version of a program is being used, check executable location like:

where gcc

You may need to reorder directories in your Windows Path variable, for example GNU Octave may need to be moved lower in the Path list.

When packages are installed in MinGW64, you must run from from MSYS2 MinGW64 terminal, or have added MSYS2 MinGW64 to your Command Prompt Path (in step 5 above).

Notes

Update MiKTeX packages to fix Windows LaTeX errors

Update MiKTeX packages to avoid issues with compiling documents. This is even true for a fresh MiKTeX install.

  1. Start → MiKTeX Console → Updates → Check for updates
  2. if updates available, click Update now

Anytime you get an unexpected error, try updating MiKTeX packages.

Add missing LaTeX fonts

Missing LaTeX fonts can be added via TexLive or MikTeX. Using the symbolic fonts is as easy as:

\documentclass[a4paper,12pt]{article}

\usepackage{fontawesome5}

\faGithub This is a GitHub logo.

You may need to use XeLaTeX in some cases.

Note that in LaTeX we use fontawesome5 instead of obsolete fontawesome, which is old version 4. FontAwesome5 is in TeXLive 2018, which on Ubuntu requires Ubuntu ≥ 18.10 or use a PPA.

Linux

Many Linux systems use TeXLive. Here’s how to get a couple popular symbolic fonts (logos for email, phone, GitHub, etc.)

marvosym

apt install texlive-fonts-recommended

fontawesome5 (for popular emoji and website icons):

apt install texlive-fonts-extra

Windows

Windows LaTeX users typically use MikTeX. MiKTeX will automatically install packages as needed. If that isn’t happening, check:

Start → MiKTeX Console → Settings → General → “Always install missing packages on the fly”

Otherwise, manually search for package name and install:

Start → MiKTeX Console → Packages

Install Intel Compilers for C, C++ and Fortran

Related: Use Intel compilers with cmake, make and load libraries


The Intel Parallel Studio XE 2020 suite includes compliers named:

  • Windows: icl (C, C++) and ifort (Fortran)
  • Linux / MacOS: icc ( C ) icpc (C++) ifort (Fortran)

Parallel Studio also includes Intel MPI compiler wrappers (on Windows and Linux):

  • mpiicc ( C )
  • mpicxx (C++)
  • mpiifort (Fortran)

OS-specific considerations include:

  • MacOS: MPICH is available instead of Intel MPI or OpenMPI with Intel compilers
  • Windows: Intel compilers on Windows require Microsoft Visual Studio of appropriate version installed with C++ extensions as well.
  • Windows: MPI has special considerations particular to Windows only.

Install

For all OS, sudo / administrator is not needed or desired to install Intel compilers. Just install under your home directory or common directory.

  1. Download the Parallel Studio XE Cluster Edition Composer Suite (online/custom installer, not the entire 4 GB file). Get the CLUSTER / “all tools” version, NOT just the Fortran version or you will NOT have MPI!
  2. Extract and run the installer.

    • Linux / MacOS: install to a permissions-aware drive (e.g. Ext4), NOT a non-permissions filesystem like ExFAT, or the compiler will silently fail to install.
    • Windows: can install to non-permissions-aware drive (ExFAT is OK if desired)
  3. Select options similar to as below, adding PGI compiler if you use it (can add later if needed):

    • architecture: Intel64 (IA-32 not used)
    • Intel C++ Compiler
    • Intel Fortran Compiler
    • Intel MKL for C/C++
    • Intel MKL for Fortran
    • LAPACK 95
    • Intel MPI Library
    • GNU GDB

Load environment

  • Linux: create a file ~/intel.sh with contents like:
. ~/intel/bin/compilervars.sh intel64
. ~/intel/mkl/bin/mklvars.sh intel64

Enable the Intel compilers each time you want to use them with:

. ~/intel.sh
  • Windows: create a file %userprofile%\intel.bat containing like (modify slightly to match your install path):
D:\IntelSWTools\compilers_and_libraries_2019.3.203\windows\bin\compilervars.bat intel64

set FC=ifort
set CC=icl
set CXX=icl

Intel MKL

If you installed MKL prior to the Intel Compiler, be sure to select Intel MKL or it will uninstall your preexisting MKL! The Intel MKL options you may want for a desktop PC include

  • Intel MKL core libraries for C/C++
  • Intel TBB threading support
  • GNU C/C++ compiler support

  • Intel MKL core libraries for Fortran

  • GNU Fortran compiler support

  • Fortran 95 interfaces for BLAS and LAPACK

CMake Intel compilers

FC=ifort CC=icc CXX=icpc cmake ..

To switch back to GNU or other compiler, you have to remove the CMake cached files. Using Intel compilers and libraries with CMake is quite straightforward and easy to switch between compilers.

Why use Intel compilers

To understand what Intel compilers can do for scaling your program from the desktop to HPC–or merely improve performance on your laptop, see Intel Compiler new user guide