Delft 3Dを始めてみよう(1)

海岸工学

Delft 3Dを始めてみよう

https://oss.deltares.nl/web/delft3d/get-started

リンク先にあるとおり、以下のステップにしたがって進めていきましょう。

  1. Registration
  2. Prerequisites - required software
  3. Download the source code
  4. Compile the source code
  5. Run a calculation
  6. Known issues

1. Registration

メールアドレスとパスワードを設定して完了です。

2. Prerequisites - required software

私はWindows版ではなくLinux　Ubuntu　18.04を使います。以下のパッケージが必要なようですのでひとつずつ見ていきましょう。

  • Subversion client
  • GNU Autotools (use Package Manager)
  • GNU Libtool
  • GNU C++ compiler (use Package Manager)
  • expat-devel (use Package Manager)
  • GNU Fortran compiler 4.9.1
  • Mpich, version 3.1.2 or higher.
  • Lex & Yacc
  • OpenSSL
  • libreadline6-dev
  • Ruby
  • NetCDF: netcdf-c-4.6.1 or above, netcdf-fortran-4.4.5 or above, with hdf5 enabled
  • uuid-dev package

Subversion client

シンプルにコマンドラインのsvnクライアントをインストールしましょう。

vagrant@ubuntu-bionic:~$ sudo apt install subversion
...
vagrant@ubuntu-bionic:~$ svn --version
svn, version 1.9.7 (r1800392)
   compiled Mar 28 2018, 08:49:13 on x86_64-pc-linux-gnu

インストールできました。

GNU Autotools (use Package Manager).

AutoconfとAutomakeをインストールします。動作が検証されているのはAutoconf version 2.68、 Automake 1.11.1でAutoconf versionの2.63は"unknown language: Fortran"というエラーメッセージが起こるバグがあるので使わないようにとありますね。普通にaptでインストールしてversionを見てみましょう。

vagrant@ubuntu-bionic:~$ sudo apt install automake autoconf
...
vagrant@ubuntu-bionic:~$ automake --version
automake (GNU automake) 1.15.1
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv2+: GNU GPL version 2 or later <http://gnu.org/licenses/gpl-2.0.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Tom Tromey <tromey@redhat.com>
       and Alexandre Duret-Lutz <adl@gnu.org>.
vagrant@ubuntu-bionic:~$ autoconf --version
autoconf (GNU Autoconf) 2.69
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+/Autoconf: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>, <http://gnu.org/licenses/exceptions.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by David J. MacKenzie and Akim Demaille.

Autoconf version 2.69、 Automake 1.15.1がインストールされました。検証されたバージョンよりも新しいので大丈夫でしょう。

GNU Libtool

libtool、libtool-devをインストールします。

vagrant@ubuntu-bionic:~$ sudo apt install libtool 
...
Setting up libtool (2.4.6-2) ...
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
Processing triggers for libc-bin (2.27-3ubuntu1) ...
vagrant@ubuntu-bionic:~$

検証済みの2.4.2より新しい2.4.6がインストールされました。

GNU C++ compiler (use Package Manager), version 3.4.6.

g++をインストールします。

sudo apt install g++
...
vagrant@ubuntu-bionic:~$ g++ --version
g++ (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0

指定されているバージョン3.4.6よりだいぶ新しいバージョンですがひとまず先に進みましょう。

expat-devel (Expat is a library to develop XML applications)(use Package Manager)

Ubuntuではパッケージ名がlibexpat1-devのようです。

vagrant@ubuntu-bionic:~$ sudo apt install libexpat1-dev
...
Setting up libexpat1-dev:amd64 (2.2.5-3ubuntu0.2) ...
vagrant@ubuntu-bionic:~$

GNU Fortran compiler 4.9.1.

vagrant@ubuntu-bionic:~$ sudo apt install gfortran
...
vagrant@ubuntu-bionic:~$ gfortran --version
GNU Fortran (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0

8.x.x以降のバージョンは使わないようにとあります。7.5.0なので大丈夫でしょう。

Mpich, version 3.1.2 or higher.

mpichとDelft3Dは同じバージョンのC++、Fortranコンパイラを使うようにとあるのでソースからコンパイルする必要がありそうでう。ひとまずaptでインストールして、うまく行かなかったらソースからコンパイルしましょう。

vagrant@ubuntu-bionic:~$ sudo apt install mpich
...
vagrant@ubuntu-bionic:~$ mpiexec --version
HYDRA build details:
    Version:                                 3.3a2
    Release Date:                            Sun Nov 13 09:12:11 MST 2016
    CC:                              gcc   -Wl,-Bsymbolic-functions -Wl,-z,relro 
    CXX:                             g++   -Wl,-Bsymbolic-functions -Wl,-z,relro 
    F77:                             gfortran  -Wl,-Bsymbolic-functions -Wl,-z,relro 
    F90:                             gfortran  -Wl,-Bsymbolic-functions -Wl,-z,relro 
    Configure options:                       '--disable-option-checking' '--prefix=/usr' '--build=x86_64-linux-gnu' '--includedir=${prefix}/include' '--mandir=${prefix}/share/man' '--infodir=${prefix}/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-silent-rules' '--libdir=${prefix}/lib/x86_64-linux-gnu' '--libexecdir=${prefix}/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--disable-dependency-tracking' '--with-libfabric' '--enable-shared' '--enable-fortran=all' '--disable-rpath' '--disable-wrapper-rpath' '--sysconfdir=/etc/mpich' '--libdir=/usr/lib/x86_64-linux-gnu' '--includedir=/usr/include/mpich' '--docdir=/usr/share/doc/mpich' '--with-hwloc-prefix=system' '--enable-checkpointing' '--with-hydra-ckpointlib=blcr' 'CPPFLAGS= -Wdate-time -D_FORTIFY_SOURCE=2 -I/build/mpich-O9at2o/mpich-3.3~a2/src/mpl/include -I/build/mpich-O9at2o/mpich-3.3~a2/src/mpl/include -I/build/mpich-O9at2o/mpich-3.3~a2/src/openpa/src -I/build/mpich-O9at2o/mpich-3.3~a2/src/openpa/src -D_REENTRANT -I/build/mpich-O9at2o/mpich-3.3~a2/src/mpi/romio/include' 'CFLAGS= -g -O2 -fdebug-prefix-map=/build/mpich-O9at2o/mpich-3.3~a2=. -fstack-protector-strong -Wformat -Werror=format-security -O2' 'CXXFLAGS= -g -O2 -fdebug-prefix-map=/build/mpich-O9at2o/mpich-3.3~a2=. -fstack-protector-strong -Wformat -Werror=format-security -O2' 'FFLAGS= -g -O2 -fdebug-prefix-map=/build/mpich-O9at2o/mpich-3.3~a2=. -fstack-protector-strong -O2' 'FCFLAGS= -g -O2 -fdebug-prefix-map=/build/mpich-O9at2o/mpich-3.3~a2=. -fstack-protector-strong -O2' 'build_alias=x86_64-linux-gnu' 'MPICHLIB_CFLAGS=-g -O2 -fdebug-prefix-map=/build/mpich-O9at2o/mpich-3.3~a2=. -fstack-protector-strong -Wformat -Werror=format-security' 'MPICHLIB_CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' 'MPICHLIB_CXXFLAGS=-g -O2 -fdebug-prefix-map=/build/mpich-O9at2o/mpich-3.3~a2=. -fstack-protector-strong -Wformat -Werror=format-security' 'MPICHLIB_FFLAGS=-g -O2 -fdebug-prefix-map=/build/mpich-O9at2o/mpich-3.3~a2=. -fstack-protector-strong' 'MPICHLIB_FCFLAGS=-g -O2 -fdebug-prefix-map=/build/mpich-O9at2o/mpich-3.3~a2=. -fstack-protector-strong' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro' 'FC=gfortran' 'F77=gfortran' 'MPILIBNAME=mpich' '--cache-file=/dev/null' '--srcdir=.' 'CC=gcc' 'LIBS=' 'MPLLIBNAME=mpl'
    Process Manager:                         pmi
    Launchers available:                     ssh rsh fork slurm ll lsf sge manual persist
    Topology libraries available:            hwloc
    Resource management kernels available:   user slurm ll lsf sge pbs cobalt
    Checkpointing libraries available:       blcr
    Demux engines available:                 poll select
vagrant@ubuntu-bionic:~$

Lex & Yacc

Lexはtokenizer、YaccはparserでSyntaxエラーの解析に使うようですね。Ubuntuではflexとbisonが相当します。これもaptでインストールしましょう。

vagrant@ubuntu-bionic:~$ sudo apt install flex bison
...
Setting up flex (2.6.4-6) ...
...
Setting up bison (2:3.0.4.dfsg-1build1) ...

OpenSSL

OpenSSLはすでにインストールされていました。

agrant@ubuntu-bionic:~$ openssl version -a
OpenSSL 1.1.1  11 Sep 2018
built on: Tue Nov 12 16:58:35 2019 UTC
platform: debian-amd64
options:  bn(64,64) rc4(16x,int) des(int) blowfish(ptr) 
compiler: gcc -fPIC -pthread -m64 -Wa,--noexecstack -Wall -Wa,--noexecstack -g -O2 -fdebug-prefix-map=/build/openssl-kxN_24/openssl-1.1.1=. -fstack-protector-strong -Wformat -Werror=format-security -DOPENSSL_USE_NODELETE -DL_ENDIAN -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DRC4_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DX25519_ASM -DPADLOCK_ASM -DPOLY1305_ASM -DNDEBUG -Wdate-time -D_FORTIFY_SOURCE=2
OPENSSLDIR: "/usr/lib/ssl"
ENGINESDIR: "/usr/lib/x86_64-linux-gnu/engines-1.1"
Seeding source: os-specific

バージョン0.9.8は使わないようにとありますが1.1.1なので大丈夫です。

"readline-devel" (RedHat/Fedora) or "libreadline6-dev" (Debian/Ubuntu)

このパッケージもaptでインストールします。

vagrant@ubuntu-bionic:~$ sudo apt install libreadline6-dev

Ruby

rubyも同様です。

vagrant@ubuntu-bionic:~$ sudo apt install ruby
...
vagrant@ubuntu-bionic:~$ ruby -v
ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux-gnu]

NetCDF: netcdf-c-4.6.1 or above, netcdf-fortran-4.4.5 or above, with hdf5 enabled

これもMpichと同様にDelft3Dと同じコンパイラバージョンを使うようにあります。ひとまずaptでインストールして進みましょう。

vagrant@ubuntu-bionic:~$ sudo apt install libnetcdf-dev libnetcdff-dev
...
vagrant@ubuntu-bionic:~$ nc-config --all

This netCDF 4.6.0 has been built with the following features: 
...
vagrant@ubuntu-bionic:~$ nf-config --all

This netCDF-Fortran 4.4.4 has been built with the following features:

uuid-dev package

これで最後です。

vagrant@ubuntu-bionic:~$ sudo apt install uuid-dev

3. Download the source code

.xで終わるユーザー名とパスワードを入力してsvnのサーバーにアクセスできました。 * https://svn.oss.deltares.nl

svn login それではsvnコマンドでソースコードをダウンロードしましょう。

vagrant@ubuntu-bionic:~$ svn checkout https://svn.oss.deltares.nl/repos/delft3d/tags/delft3d4/59659 delft3d_repository
Authentication realm: <https://svn.oss.deltares.nl:443> OSS Subversion Server
Password for 'vagrant': *******

Authentication realm: <https://svn.oss.deltares.nl:443> OSS Subversion Server
Username:
...
Checked out revision 66656.

delft3d_repositoryという新しいディレクトリにダウンロード出来ました。

vagrant@ubuntu-bionic:~$ du -h --max-depth=1 delft3d_repository/ | sort -h
12K delft3d_repository/doc
40M delft3d_repository/examples
953M    delft3d_repository/.svn
997M    delft3d_repository/src
2.0G    delft3d_repository/

合計で２GBのサイズですね。svnは昔WindowsのTurtoiseSVNを使ったことがありますがコマンドラインは初めてです。revision 66656.が最新でしょう。

srcディレクトリを見てみるとサードパーティのサイズが圧倒的に大きいですね。

vagrant@ubuntu-bionic:~/delft3d_repository/src$ du -h --max-depth=1 ./ | sort -h
140K    ./m4
160K    ./plugins_lgpl
300K    ./scripts_lgpl
1.4M    ./third_party
4.2M    ./tools_gpl
5.7M    ./utils_gpl
7.9M    ./tools_lgpl
8.1M    ./logs
45M ./engines_gpl
82M ./utils_lgpl
843M    ./third_party_open
997M    ./

拡張子別のファイル数を見てみます。さすが数値計算ソフトウェアなのでfortranが多いですね。

vagrant@ubuntu-bionic:~/delft3d_repository/src$ find . -type f | sed -n 's/..*\.//p' | sort | uniq -c | sort -nr
   1330 f90
    829 f
    681 m
    558 c
    452 h
    398 am
    228 F90
    177 dll
    162 vfproj
    153 txt
    136 cpp
    102 vcxproj
     95 inc
     88 tab
     79 exe
     78 svn-base
     78 igs
     77 bat
     76 sh
     64 so
     62 ini
     58 clrmap
     56 vcproj
     56 dsp
     54 svn
     52 m4
...

4. Compile the source code

src/READMEを読んでコンパイルするようにあるので読んでみましょう。

Compiling on Linux
==================
  0) Be sure that you have installed all prerequisites listed at https://oss.deltares.nl/web/delft3d/source-code#prerequisites
  1) In directory "src", do one of the following two options:
     Option A):
         ./autogen.sh
         cd third_party_open/kdtree2
         ./autogen.sh
         cd -
         CFLAGS='-O2' CXXFLAGS='-O2' FFLAGS='-O2' FCFLAGS='-O2' ./configure --prefix=`pwd` --with-netcdf --with-mpi --with-metis --with-petsc
         make ds-install
     Option B):
         Use script build.sh. Executing "./build.sh" will show the usage. The following example line is used to create the latest fully tested version:
         ./build.sh -intel14
         WARNING: build.sh is system dependent. You have to modify it to fit to your site!
  1b) OPTIONAL: the Delft3D-Flexible Mesh kernel is not built by default. You can do this
     in a second step as follows. In directory "src", do:
         FC=mpif90 make ds-install -C engines_gpl/dflowfm
  2) The binaries are installed in directory "src/bin", "src/lib" and "src/share" (due to "--prefix=`pwd`" and default "install")
     In "src/bin", execute:
     libtool_install.sh
     This will re-install dimr on the same location as it is, but now using libtool.
     This step has to be executed every time the binaries are installed/moved to another location.
  3) To clean (removing all files generated by autoreconf/configure/make):
     ./clean.sh
  4) Remarks:
     - Compiling on 32bit Linux platforms does not work. This is a known bug.
     - Be careful with tools/compilers located in different directories due to different revision/build numbers (e.q. /opt/mpich2-1.4.1-gcc-4.6.2)
       You have to take care of that yourself
     - Check both "common.am" and "build.sh"
     - Check http://oss.deltares.nl/web/delft3d/faq
     - When using a --prefix unequal to `pwd`, the specified directory will contain:
       - "bin", "lib" and "share" subdirectories (due to the default Linux "install")
         The examples will not run "out of the box". The runscripts must be adapted to point to the --prefix location

Option A)で行ってみましょう。

一度目のautogen.sh.

vagrant@ubuntu-bionic:~/delft3d_repository/src$ ./autogen.sh 
Preparing the Delft3D build system...please wait

Found GNU Autoconf version 2.69
Found GNU Automake version 1.15.1
Found GNU Libtool version 2.4.6

Automatically preparing build ... done

The Delft3D build system is now prepared.  To build here, run:
  ./configure
  make
vagrant@ubuntu-bionic:~/delft3d_repository/src$

kdtree2ディレクトリでautogen.sh.

vagrant@ubuntu-bionic:~/delft3d_repository/src/third_party_open/kdtree2$ ./autogen.sh 
Preparing the KDTREE 2 build system...please wait

Found GNU Autoconf version 2.69
Found GNU Automake version 1.15.1
Found GNU Libtool version 2.4.6

Automatically preparing build ... done

The KDTREE 2 build system is now prepared.  To build here, run:
  ./configure
  make
vagrant@ubuntu-bionic:~/delft3d_repository/src/third_party_open/kdtree2$

srcディレクトリでmakeの設定をします。

vagrant@ubuntu-bionic:~/delft3d_repository/src$ cd -
vagrant@ubuntu-bionic:~/delft3d_repository/src$ CFLAGS='-O2' CXXFLAGS='-O2' FFLAGS='-O2' FCFLAGS='-O2' ./configure --prefix=`pwd` --with-netcdf --with-mpi --with-metis --with-petsc
...
gfortran: error: unrecognized command line option '-showme:link'; did you mean '--force-link'?
configure: creating ./config.lt
config.lt: creating libtool
checking for the extension of Delft3D plugins... .so
./configure: line 29905: ACX_MKL: command not found
checking for PETSc... no
configure: error: Package requirements (PETSc < 3.5) were not met:

No package 'PETSc' found

Consider adjusting the PKG_CONFIG_PATH environment variable if you
installed software in a non-standard prefix.

Alternatively, you may set the environment variables PETSc_CFLAGS
and PETSc_LIBS to avoid the need to call pkg-config.
See the pkg-config man page for more details.

gfortranとPETScのエラーが気になりますね。それでもコンパイルを試してみましょう。

vagrant@ubuntu-bionic:~/delft3d_repository/src$ make ds-install
Making install in third_party
...
/bin/bash ../../../../../libtool  --tag=FC   --mode=compile mpif90 -DHAVE_CONFIG_H -I. -I../../../../..     -I../../../../../utils_lgpl/deltares_common/packages/deltares_common/src -DWITH_DELFTONLINE  -I/usr/include/hdf5/serial -I/usr/include/hdf5/serial -O2 -ffree-line-length-none -cpp -c -o io_netcdf_version.lo io_netcdf_version.F90
libtool: compile:  mpif90 -DHAVE_CONFIG_H -I. -I../../../../.. -I../../../../../utils_lgpl/deltares_common/packages/deltares_common/src -DWITH_DELFTONLINE -I/usr/include/hdf5/serial -I/usr/include/hdf5/serial -O2 -ffree-line-length-none -cpp -c io_netcdf_version.F90  -fPIC -o .libs/io_netcdf_version.o
libtool: compile:  mpif90 -DHAVE_CONFIG_H -I. -I../../../../.. -I../../../../../utils_lgpl/deltares_common/packages/deltares_common/src -DWITH_DELFTONLINE -I/usr/include/hdf5/serial -I/usr/include/hdf5/serial -O2 -ffree-line-length-none -cpp -c io_netcdf_version.F90 -o io_netcdf_version.o >/dev/null 2>&1
/bin/bash ../../../../../libtool  --tag=FC   --mode=compile mpif90 -DHAVE_CONFIG_H -I. -I../../../../..     -I../../../../../utils_lgpl/deltares_common/packages/deltares_common/src -DWITH_DELFTONLINE  -I/usr/include/hdf5/serial -I/usr/include/hdf5/serial -O2 -ffree-line-length-none -cpp -c -o netcdf_utils.lo netcdf_utils.F90
libtool: compile:  mpif90 -DHAVE_CONFIG_H -I. -I../../../../.. -I../../../../../utils_lgpl/deltares_common/packages/deltares_common/src -DWITH_DELFTONLINE -I/usr/include/hdf5/serial -I/usr/include/hdf5/serial -O2 -ffree-line-length-none -cpp -c netcdf_utils.F90  -fPIC -o .libs/netcdf_utils.o
netcdf_utils.F90:33:4:

 use netcdf
    1
Fatal Error: Can't open module file ‘netcdf.mod’ for reading at (1): No such file or directory
compilation terminated.
Makefile:545: recipe for target 'netcdf_utils.lo' failed
make[5]: *** [netcdf_utils.lo] Error 1
make[5]: Leaving directory '/home/vagrant/delft3d_repository/src/utils_lgpl/io_netcdf/packages/io_netcdf/src'
Makefile:493: recipe for target 'install-recursive' failed
make[4]: *** [install-recursive] Error 1
make[4]: Leaving directory '/home/vagrant/delft3d_repository/src/utils_lgpl/io_netcdf/packages/io_netcdf'
Makefile:493: recipe for target 'install-recursive' failed
make[3]: *** [install-recursive] Error 1
make[3]: Leaving directory '/home/vagrant/delft3d_repository/src/utils_lgpl/io_netcdf/packages'
Makefile:493: recipe for target 'install-recursive' failed
make[2]: *** [install-recursive] Error 1
make[2]: Leaving directory '/home/vagrant/delft3d_repository/src/utils_lgpl/io_netcdf'
Makefile:506: recipe for target 'install-recursive' failed
make[1]: *** [install-recursive] Error 1
make[1]: Leaving directory '/home/vagrant/delft3d_repository/src/utils_lgpl'
Makefile:554: recipe for target 'install-recursive' failed
make: *** [install-recursive] Error 1

netcdfのエラーでストップしてしまいました。検索してみるとnetcdfをソースからコンパイルしていないのが原因のようです。

https://oss.deltares.nl/web/delft3d/coast-/-estuary/-/message_boards/message/1914702