Compilation Guideline of OpenSeeMP on Linux Machines

2013年01月27日 14:18:05 苏内容
  标签: OpenSees/Linux
阅读:7328

Compilation Guideline of OpenSeeMP on Linux Machines

This guide is developed in a systematic manner assuming ActiveTcl, MPICH2 and other packages are not pre-installed in the system. But even if those are available slight modification will enable us to compile OpenSeesMP.

The following tarballs are used to compile OpenSeesMP in this manual. This instruction only works with OpenSees 2.3.2.2 version(svn code – 4985). Package’s version has importance for compatibility issues.

 

Required Packages

ActiveTcl8.5.11.1.295590-linux-x86_64-threaded.tar.gz
db-5.3.21.NC.tar.gz
lapack-3.4.1.tgz
metis-4.0.3.tar.gz
mpiblacs-patch03.tgz
mpiblacs.tgz –files inside needs to be overwritten by files of mpiblacs-patch03.tgz
mpich2-1.1.tar.gz
MUMPS_4.10.0.tar.gz
openssl-1.0.1c.tar.gz
ParMetis-3.2.0.tar.gz
scalapack-2.0.2.tgz
xblas-1.0.248.tar.gz

You can find the following packages in internet, all of them are free, some of them require an email to the developer.

 

Some Terminal Commands

Some knowledge of bash commands are necessary, parallel machines often do not have graphical user interface.

mkdir mydir - makes a directory named mydir
chmod 777 a.txt - change write permission of files to full permission
nano a.txt - opens a.txt for editing
cd mydir - gets inside mydir
cd .. - gets out mydir to parent directory
which mpd - checks which mpd is used and returns the location

Some system environment packages

Before compiling any of these packages we need to perform the following tasks - (apt-get command is for ubuntu machines for redhat machines we can use yum install) sudo in and type the following:

mkdir lib
mkdir bin
sudo apt-get update 
sudo apt-get install build-essential
sudo apt-get install subversion
svn co svn://opensees.berkeley.edu/usr/local/svn/OpenSees/trunk@4985 OpenSees

Use-

chmod 777 OpenSees 

To change the editing permission of the directory.

sudo apt-get install make
sudo apt-get install gcc
sudo apt-get install g++
sudo apt-get install gfortran
sudo apt-get install mysql

Compilation of Necessary Packages

After completion of the above mentioned process we need to customize compile and install some packages, which should be done in a ordered manner-

To be systematic and implementation on cluster HPC we will create a directory in home called “PARALLEL”. And put all the tarballs required extracts – in simple named folder like scalapack for sclalapack-2.0.2. Hence we get all packs in one master folder.

The directory structure should be setup as following-

HOME=/home/yourusername
PARALLEL=$(HOME)/PARALLEL
level0 |   level1        |level2

HOME   |   PARALLEL      |blacs
              		 |mpich2
              		 |lapack
             		 |scalapack
             		 |blas
             		 |xblas
             		 |metis
             		 |parmetis
             		 |mumps
              		 |---
Fig – Example directory structure

 

Install Activetcl 8.5.11 - x86-64 (Requires Admin Access)

Untar the installation package, go inside the directory , type –

install.sh 

By default tcl will be installed in the following directory –

TCLdir=/opt/ActiveTcl-8.5

TCL Directories to remember

TCL_BIN=$(TCLdir)/bin

TCL_INC=-I$(TCLdir)/include

TCL_LIB=-L$(TCLdir)/lib -ltcl8.5 -ltk8.5

 

We need to remember one thing!! tcl.init file location $(tcldir)/lib/tcl8.5/tcl.int

 

Install mpich2-1.1 for compatibility issues with mumps

Configuration steps ---

a. Setting prefix/ installation directory – for csh and tcsh:

./configure --prefix=/home/yourusername/PARALLEL/mpich2-install |& tee c.txt

for bash and sh:

./configure --prefix=/home/yourusername/PARALLEL/mpich2-install 2>&1 | tee c.txt

b. Build MPICH2:

for csh and tcsh:

make |& tee m.txt

for bash and sh:

make 2>&1 | tee m.txt

c. Install the MPICH2 commands:

for csh and tcsh:

make install |& tee mi.txt

for bash and sh:

make install 2>&1 | tee mi.txt

d. Add the bin subdirectory of the installation directory to your path:

for csh and tcsh:

setenv PATH=/home/yourusername/PARALLEL/mpich2-install/bin:$PATH

for bash and sh:

export PATH=/home/yourusername/PARALLEL/mpich2-install/bin:$PATH

Make permanent change to system variable to recognise mpich2. Go to user home (don’t confuse with OpenSees compilation home). Add the line of above section d. at the last line.

chmod 777 .bashrc
nano .bashrc

Run the following command to se wheather system variables are changed or not -

which mpd
which mpiexec
which mpirun

Run this command to check everything is working ok. e. mpd run Begin by placing in your home directory a file named .mpd.conf (/etc/mpd.conf if root), containing the line

secretword=<secretword>

where <secretword> is a string known only to yourself. It should NOT be your normal Unix password. Make this file readable and writable only by you:

chmod 600 .mpd.conf

The first sanity check consists of bringing up a ring of one mpd on the local machine, testing one mpd command, and bringing the "ring" down.

mpd &
mpdtrace
mpdallexit

The output of mpdtrace should be the hostname of the machine you are running on. The mpdallexit causes the mpd daemon to exit. If you have problems getting the mpd ring established, see the Installation Guide of MPICH2 for instructions on how to diagnose problems with your system configuration that may be preventing it.

We will notedown the important directory location of MPICH2-

MPIdir=$(HOME)/PARALLEL/mpich2-install

MPI_BIN=$(MPIdir)/bin

MPI_INC=-I$(MPIdir)/include

MPI_LIB=-L$(MPIdir)/lib

 

Use the full directory of all the address of string like $(HOME), $(MPIdir) as system will not recognise such strings used in following 3 to 12 step. After editing the makefiles run relevant make commands for each individual packages.

Install BLAS

Edit the make makefile –

PLAT =
FORTRAN  = $(MPI_BIN)/mpif90
LOADER   = $(MPI_BIN)/mpif90

Install XBLAS

Edit the make makefile –

CC = $(MPI_BIN)/mpicc
EXTRA_LIBS =  -lm

Search for the location of libm.a if not found automatically, enter the location of the file in place of -lm.

 

Install LAPACK 3.4.1

Edit the make makefile –

FORTRAN	= $(MPI_BIN)/mpif90
LOADER	= $(MPI_BIN)/mpif90
USEXBLAS	= Yes
XBLASLIB	=$(PARALLEL)/xblas/libxblas.a
BLASLIB	= $(PARALLEL)/blas/blas.a

Install mpiBLACS

Edit the following in the makefile –

COMMLIB =
PLAT =
BTopdir =$(HOME)/PARALLEL/blacs
MPIdir =$(HOME)/PARALLEL/mpich2-install
MPILIBdir = $(MPIdir)/lib
MPIINCdir = $(MPIdir)/include
MPILIB = $(MPILIBdir)/libmpich.a

SYSINC = -I$(MPIINCdir)

INTFACE = -DAdd_***
  • The Fortran 77 to C interface to be used. If you are unsure of the correct setting for your platform, compile and run BLACS/INSTALL/xintface. Choices are: Add_, NoChange, UpCase, or f77IsF2C.

Go to blacs/INSTALL/EXE

make xintface
./xintface #to check interface type use similar command to check others.
TRANSCOMM = -DUseMpich***
WHATMPI = ***
SYSERRORS =
  • above parameterhas to be identified from running all the executable in BLACS/INSTALL/EXE directory
F77            = $(MPI_BIN)/mpif77
  CC             = $(MPI_BIN)/mpicc
  CCLOADER       = $(MPI_BIN)/mpicc

 

Install SCALAPACK 2.0.2

Edit the make makefile –

 

CDEFS		= -DAdd_ # found from mpiBLACS section
FC		= $(MPI_BIN)/mpifc 
CC		= $(MPI_BIN)/mpicc
BLASLIB	= $(PARALLEL)/blas/blas.a
LAPACKLIB	= $(PARALLEL)/lapack/liblapack.a

 

Install metis 4.x.x

Edit the make makefile –

CC            = $(MPI_BIN)/mpicc

Install parmetis 3.x.x

Edit the make makefile –

CC		= $(MPI_BIN)/mpicc
INCDIR 	= $(MPIdir)/include
LD		= $(MPI_BIN)/mpicc

 

Install MUMPS_4.10.0

Edit the makefile like following -

LMETISDIR = $(PARALLEL)/parmetis
IMETIS    = -I$(LMETISDIR)
LMETIS    = -L$(LMETISDIR) -lparmetis –lmetis
ORDERINGSF  = -Dmetis -Dpord –Dparmetis
PLAT    =
CC      = $(MPI_BIN)/mpicc 
FC      = $(MPI_BIN)/mpif90
FL      = $(MPI_BIN)/mpif90
##FOR SCLAPACK BLACS SANDWITCHING IS NECESSARY DON’T REMOVE UNLESS YOU KNOW WHAT YOU ARE DOING
SCALAP  = -L$(PARALLEL)/scalapack -lscalapack \
       $(PARALLEL)/blacs/LIB/blacs.a \
       $(PARALLEL)/blacs/LIB/blacsCinit.a \
       $(PARALLEL)/blacs/LIB/blacs.a \
       $(PARALLEL)/blacs/LIB/blacsF77init.a\
       $(PARALLEL)/blacs/LIB/blacs.a \
       $(PARALLEL)/lapack/liblapack.a
INCPAR  = -I$(MPIdir)/include
LIBPAR  = $(SCALAP) \
          -L$(MPIdir)/lib

LIBBLAS = $(PARALLEL)/blas/blas.a
CDEFS = -DAdd_# identified earlier by mpiBLACS check in previous section

BerkeleyDB5.3

Perform the following command-

cd build_unix
../dist/configure
make
make install

You can use prefix to install at your preferred directory, if you don't have enough permission. Be careful to notedown the installation directory.

openssl 1.0.1

Use ./configure and track the location of include directory we will need it later

  • For other installed files, find location using whereis command
libdl
libm
libssl
libdb
libcrypto
libgfortran

If not found automatically, try to find manually from all library folders of machine.

Edit The Makefile.def

Copy all the text from following section. Make a new Makefile.def in your machine. Edit the sections which are marked to be edited carefully. Watch this comment in the file and adjust to your machines specificity -"!!!!!!!!!!!!!USE MACHINES SPECIFIC LOCATIONS!!!!!!!!"

 

############################################################################
#
#  Program:  OpenSees
#
#  Purpose:  A Top-level Makefile to create the libraries needed
#	     to use the OpenSees framework. Works on Linux version 6.1
#            and below.
#
#  Written: fmk 
#  Created: 10/99
#
#  Send bug reports, comments or suggestions to fmckenna@ce.berkeley.edu
#
############################################################################
 
# copy /opt/ActiveTcl-8.5/lib/tcl8.5/init.tcl to tcl library location
###################################################################
# CHANGE THIS SECTION AND HAVE A LOOK AT THE SECTION HAVING 
#!!!!!!!!!!!!!USE MACHINES SPECIFIC LOCATIONS!!!!!!!!!!!!! MARK
###################################################################
HOME = /home/ciminelli/khaled
TCLdir = /app/ActiveTcl-8.5
BERKLEYDir=$(HOME)/bdb
MPIdir = $(HOME)/PARALLEL/mpich2-install
 
##################################################################
PARALLELdir=$(HOME)/PARALLEL
##TCL
TCL_BIN = $(TCLdir)/bin
TCL_INC = $(TCLdir)/include
TCL_LIB = -L$(TCLdir)/lib
######################################################################################
 
# %---------------------------------%
# |  SECTION 1: PROGRAM             |
# %---------------------------------%
#
# Specify the location and name of the OpenSees interpreter program
# that will be created (if this all works!)
#PROGRAMMING_MODE = SEQUENTIAL
#PROGRAMMING_MODE = PARALLEL
PROGRAMMING_MODE = PARALLEL_INTERPRETERS
 
OpenSees_PROGRAM = $(HOME)/bin/OpenSees
 
ifeq ($(PROGRAMMING_MODE), PARALLEL)
OpenSees_PROGRAM = $(HOME)/bin/OpenSeesSP
endif
ifeq ($(PROGRAMMING_MODE), PARALLEL_INTERPRETERS)
OpenSees_PROGRAM = $(HOME)/bin/OpenSeesMP
endif
 
# %---------------------------------%
# |  SECTION 2: MAKEFILE CONSTANTS  |
# %---------------------------------%
#
# Specify the constants the are used as control structure variables in the Makefiles.
 
OPERATING_SYSTEM = LINUX
#DEBUG_MODE = DEBUG, NO_DEBUG
DEBUG_MODE = NO_DEBUG
#RELIABILITY = YES_RELIABILITY
RELIABILITY = NO_RELIABILITY
GRAPHICS = NONE
 
# %---------------------------------%
# |  SECTION 3: PATHS               |
# %---------------------------------%
#
# Note: if vendor supplied BLAS and LAPACK libraries or if you have
# any of the libraries already leave the directory location blank AND
# remove the directory from DIRS.
 
 
FE	= $(HOME)/OpenSees/SRC
AMDdir       	= $(HOME)/OpenSees/OTHER/AMD
BLASdir      =
CBLASdir     = $(HOME)/OpenSees/OTHER/CBLAS
LAPACKdir    =
SUPERLUdir   = $(HOME)/OpenSees/OTHER/SuperLU_4.1/SRC
ARPACKdir    = $(HOME)/OpenSees/OTHER/ARPACK
UMFPACKdir   = $(HOME)/OpenSees/OTHER/UMFPACK
METISdir       =
ITPACKdir = $(HOME)/OpenSees/OTHER/ITPACK
SUPERLU_DISTdir   = $(HOME)/OpenSees/OTHER/SuperLU_DIST_2.5/SRC
 
DIRS        = $(AMDdir) $(CBLASdir) $(ITPACKdir)\
	$(SUPERLUdir) $(SUPERLU_DISTdir) $(ARPACKdir) $(UMFPACKdir) $(FE) 
#DIRS        = $(AMDdir) $(BLASdir) $(CBLASdir) $(LAPACKdir)  $(ITPACKdir)\
#	$(SUPERLUdir) $(SUPERLU_DISTdir) $(ARPACKdir) $(UMFPACKdir) $(METISdir) $(FE) 
# %-------------------------------------------------------%
# | SECTION 4: LIBRARIES                                  |
# |                                                       |
# | The following section defines the libraries that will |
# | be created and/or linked with when the libraries are  | 
# | being created or linked with.                         |
# %-------------------------------------------------------%
#
# Note: if vendor supplied BLAS and LAPACK libraries leave the
# libraries blank. You have to get your own copy of the tcl/tk 
# library!! 
#
# Note: For libraries that will be created (any in DIRS above)
# make sure the directory exsists where you want the library to go!
#Dir definition
 
 
FE_LIBRARY      = $(HOME)/lib/libOpenSees.a
NDARRAY_LIBRARY = $(HOME)/lib/libndarray.a # BJ_UCD jeremic@ucdavis.edu
MATMOD_LIBRARY  = $(HOME)/lib/libmatmod.a  # BJ_UCD jeremic@ucdavis.edu
BJMISC_LIBRARY  = $(HOME)/lib/libBJmisc.a  # BJ_UCD jeremic@ucdavis.edu
LAPACK_LIBRARY  = $(PARALLELdir)/lapack/liblapack.a #!!!!!!!!!!!!!!!!!!!!!
BLAS_LIBRARY    = $(PARALLELdir)/blas/blas.a #!!!!!!!!!!!!!!!!!!!!!!!!!!
SUPERLU_LIBRARY = $(HOME)/lib/libSuperLU.a
CBLAS_LIBRARY   = $(HOME)/lib/libCBlas.a
ARPACK_LIBRARY  = $(HOME)/lib/libArpack.a
AMD_LIBRARY  = $(HOME)/lib/libAMD.a
UMFPACK_LIBRARY = $(HOME)/lib/libUmfpack.a
ITPACK_LIBRARY = $(HOME)/lib/libItpack.a
METIS_LIBRARY   = $(PARALLELdir)/parmetis/libparmetis.a #PARMETIS USED
#METIS_LIBRARY   = $(HOME)/lib/libMetis.a 
TCL_LIBRARY = $(TCLdir)/lib/libtcl8.5.a
BLITZ_LIBRARY =
# $(HOME)/blitz/lib/libblitz.a#
 
DISTRIBUTED_SUPERLU_LIBRARY = $(HOME)/lib/libDistributedSuperLU.a
GRAPHIC_LIBRARY     = 
 
ifeq ($(RELIABILITY), YES_RELIABILITY)
RELIABILITY_LIBRARY = $(HOME)/lib/libReliability.a
else
RELIABILITY_LIBRARY = 
endif
 
# WATCH OUT .. These libraries are removed when 'make wipe' is invoked.
WIPE_LIBS	= $(FE_LIBRARY) \
		$(CBLAS_LIBRARY) \
		$(SUPERLU_LIBRARY) \
		$(DISTRIBUTED_SUPERLU_LIBRARY) \
		$(ARPACK_LIBRARY) \
		$(UMFPACK_LIBRARY) \
	        	$(NDARRAY_LIBRARY) \
		$(MATMOD_LIBRARY) \
		$(ITPACK_LIBRARY)\
		$(AMD_LIBRARY)
 
# %---------------------------------------------------------%
# | SECTION 5: COMPILERS                                    |
# |                                                         |
# | The following macros specify compilers, linker/loaders, |
# | the archiver, and their options.  You need to make sure |
# | these are correct for your system.                      |
# %---------------------------------------------------------%
 
# ################################################### 
# # Compilers
# ################################################### 
 
MPI_BIN = $(MPIdir)/bin
MPI_INC = -I$(MPIdir)/include
MPI_LIB = -L$(MPIdir)/lib
 
CC++            = $(MPI_BIN)/mpicxx
CC              = $(MPI_BIN)/mpicc
FC              = $(MPI_BIN)/mpif90
FC90            = $(MPI_BIN)/mpif90
FC77		= $(MPI_BIN)/mpif90
FORTRAN         = $(FC)
 
LINKER          =  $(CC++)
 
#FORTRAN	= /usr/bin/gfortran
#CC++	= /usr/bin/g++
#CC      = /usr/bin/gcc
#FC	= /usr/bin/gfortran
#FORTRAN	= /usr/bin/gfortran
#LINKER	= /usr/bin/mpicxx 
AR		= ar 
ARFLAGS		= cqls
RANLIB		= ranlib
RANLIBFLAGS     =
 
GRAPHIC_FLAG = -D_NOGRAPHICS
PROGRAMMING_FLAG =
 
ifeq ($(PROGRAMMING_MODE), PARALLEL)
PROGRAMMING_FLAG = -D_PARALLEL_PROCESSING
endif
 
ifeq ($(PROGRAMMING_MODE), PARALLEL_INTERPRETERS)
PROGRAMMING_FLAG = -D_PARALLEL_INTERPRETERS
endif
 
#RELIABILITY_FLAG = -D_RELIABILITY
RELIABILITY_FLAG =
ifeq ($(RELIABILITY), YES_RELIABILITY)
RELIABILITY_FLAG = -D_RELIABILITY
else
RELIABILITY_FLAG =
endif
 
#DEBUG_FLAG = -D_G3DEBUG 
#DEBUG_FLAG = -g -p -pg 
#DEBUG_FLAG = -p -g
DEBUG_FLAG = 
ifeq ($(DEBUG_MODE), DEBUG)
DEBUG_FLAG = -D_G3DEBUG 
else
DEBUG_FLAG =
endif
MUMPS_FLAG =
 
PETSC_FLAG =
 
OPT_FLAG = -O2
#OPT_FLAG = -O0
 
#COMP_FLAG = -DMPICH_IGNORE_CXX_SEEK 
COMP_FLAG =
 
C++FLAGS	= $(GRAPHIC_FLAG) $(RELIABILITY_FLAG) $(DEBUG_FLAG) $(OPT_FLAG) $(COMP_FLAG) $(PROGRAMMING_FLAG)  $(PETSC_FLAG) $(MUMPS_FLAG) -D_TCL85 -D_BLAS -D_LINUX -D_UNIX  
 
CFLAGS          =  $(GRAPHIC_FLAG) $(RELIABILITY_FLAG) $(DEBUG_FLAG)$(PROGRAMMING_FLAG) $(OPT_FLAG) $(COMP_FLAG) -D_TCL85 -D_BLAS
FFLAGS          =  $(OPT_FLAG) $(COMP_FLAG)
LINKFLAGS       = -Wl,-rpath
 
#C++FLAGS         = -D_LINUX -D_UNIX  $(GRAPHIC_FLAG) $(RELIABILITY_FLAG) $(DEBUG_FLAG) $(OPT_FLAG) $(COMP_FLAG)\
		#$(PROGRAMMING_FLAG)  $(PETSC_FLAG) $(MUMPS_FLAG) \
		#-D_TCL85 -D_BLAS
 
#CFLAGS          =  $(GRAPHIC_FLAG) $(RELIABILITY_FLAG) $(DEBUG_FLAG)$(PROGRAMMING_FLAG) $(OPT_FLAG) $(COMP_FLAG) -D_TCL85 -D_BLAS
#FFLAGS          =  $(OPT_FLAG) $(COMP_FLAG)
#LINKFLAGS       = -Wl,-rpath
 
 
 
# Misc
MAKE		= make
CD              = cd
ECHO            = echo
RM              = rm
RMFLAGS         = -f
SHELL           = /bin/sh
 
# %---------------------------------------------------------%
# | SECTION 6: COMPILATION                                  |
# |                                                         |
# | The following macros specify the macros used in         |
# | to compile the source code into object code.            |
# %---------------------------------------------------------%
 
.SUFFIXES:
.SUFFIXES:	.C .c .f .f77 .f90 .cpp .o .cpp
 
#
# %------------------%
# | Default command. |
# %------------------%
#
.DEFAULT:
	@$(ECHO) "Unknown target $@, try:  make help"
#
# %-----------------------------------------------%
# |  Command to build .o files from source files. |
# %-----------------------------------------------%
.cpp.o: 
	@$(ECHO) Making $@ from $< $@ with $(CC++) $(C++FLAGS) $(INCLUDES) -c $<
	@$(CC++) $(C++FLAGS) $(INCLUDES) -c $<
 
 
.C.o:
	@$(ECHO) Making $@ from $<
	$(CC++) $(C++FLAGS) $(INCLUDES) -c $<
 
.c.o:
	@$(ECHO) Making $@ from $<
	$(CC) $(CFLAGS) -c  $< 
 
.f.o:      
	@$(ECHO) Making $@ from $<
	$(FC) $(FFLAGS) -c  $< 
 
.f77.o:      
	@$(ECHO) Making $@ from $<
	$(FC77) $(FFLAGS) -c $< 
 
.f90.o:      
	@$(ECHO) Making $@ from $<
	$(FC90) $(FFLAGS) -c $<
 
# %---------------------------------------------------------%
# | SECTION 7: OTHER LIBRARIES                              |
# |                                                         |
# | The following macros specify other libraries that must  |
# | be linked with when creating executables. These are     |
# | platform specific and typically order does matter!!     |
# %---------------------------------------------------------%
MACHINE_LINKLIBS  = $(MPI_LIB)\
-L$(BASE)/lib \ 				#!!!!!!!!!!!!!USE MACHINES SPECIFIC LOCATIONS!!!!!!!!
-L$(HOME)/lib \					#!!!!!!!!!!!!!USE MACHINES SPECIFIC LOCATIONS!!!!!!!!
-L/usr/lib64 \					#!!!!!!!!!!!!!USE MACHINES SPECIFIC LOCATIONS!!!!!!!!
-L/usr/lib64  -ldl -lm \		#!!!!!!!!!!!!!USE MACHINES SPECIFIC LOCATIONS!!!!!!!!
/usr/lib/gcc/x86_64-redhat-linux/4.4.4/libgfortran.a	#!!!!!!!!!!!!!USE MACHINES SPECIFIC LOCATIONS!!!!!!!!
# PETSC
HAVEPETSC = NO
PETSCINC =
PETSC_LIB = 
 
ifeq ($(PROGRAMMMING_MODE), SEQUENTIAL)
HAVEPETSC = NO
endif
 
 
ifeq ($(HAVEPETSC), YES)
 
PETSC = YES
PETSC_FLAG = -D_PETSC
PETSC_DIR = $(HOME)/OpenSees/parlibs/petsc-3.3-p2/1#customized
PETSC_ARCH = $(HOME)/OpenSees/parlibs/petsc-3.3-p2/arch-linux2-c-debug#customized
BOPT = O
 
PETSC_INC = -I$(PETSC_DIR)/include \
		-I$(PETSC_ARCH)/include \
 
 
# -D_PETSC 
# 
 
PETSC_LIB = $(FE)/system_of_eqn/linearSOE/petsc/PetscSOE.o \
	$(FE)/system_of_eqn/linearSOE/petsc/PetscSolver.o \
	$(FE)/system_of_eqn/linearSOE/petsc/PetscSparseSeqSolver.o \
	-L$(PETSC_DIR)/lib \
#-L/usr/X11/lib -lX11 -lGL
endif
 
 
HAVEMUMPS = YES
MUMPS_INCLUDE =
MUMPS_LIB = 
 
 
ifeq ($(PROGRAMMMING_MODE), SEQUENTIAL)
HAVEMUMPS = NO
endif
 
ifeq ($(HAVEMUMPS), YES)
 
MUMPS = YES
MUMPS_FLAG = -D_MUMPS
#BLAS
BLASdir = $(PARALLELdir)/blas
BLAS_LIB = $(BLASdir)/blas.a
#XBLAS
XBLAS_LIB = $(PARALLELdir)/xblas/libxblas.a
#SCALAPACK
SCALAPACK_LIB = $(PARALLELdir)/scalapack/libscalapack.a
#BLACS
BLACSdir = $(PARALLELdir)/blacs
BLACS_LIB = -L$(BLACSdir)/LIB
 
#LAPACK
LAPACKdir = $(PARALLELdir)/lapack
LAPACK_LIB = -L$(LAPACKdir)
#MUMPS
MUMPSdir = $(PARALLELdir)/mumps
MUMPSLIB = $(MUMPSdir)/lib
MUMPS_INC = $(MUMPSdir)/include
#ParMETIS
parmetis_lib = $(HOME)/PARALLEL/parmetis/libparmetis.a
metis_lib = $(HOME)/PARALLEL/parmetis/libmetis.a
metis_inc = -I$(HOME)/PARALLEL/parmetis
 
SCALAP  = $(SCALAPACK_LIB) \
	$(BLACSdir)/LIB/blacsF77init.a \
	$(BLACSdir)/LIB/blacs.a \
 	$(BLACSdir)/LIB/blacsCinit.a \
	$(LAPACKdir)/liblapack.a \
	$(XBLAS_LIB) \
	$(BLASdir)/blas.a
 
#-L/usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.5/libgfortran.a $(BLACS_LIB) $(BLAS_LIB)
#$(LAPACK_LIB) $(BLACS_LIB) $(BLAS_LIB) $(BLACS_LIB)
 
PLAT = LINUX
MUMPS_LIB = $(FE)/system_of_eqn/linearSOE/mumps/MumpsSOE.o \
	$(FE)/system_of_eqn/linearSOE/mumps/MumpsSolver.o \
	$(FE)/system_of_eqn/linearSOE/mumps/MumpsParallelSOE.o \
	$(FE)/system_of_eqn/linearSOE/mumps/MumpsParallelSolver.o \
	$(SCALAP) \
	-L$(MUMPSdir)/lib -lcmumps -ldmumps -lsmumps -lzmumps \
	-lmumps_common -lpord \
	$(parmetis_lib) $(metis_lib) \
	$(SCALAP) \
	/usr/lib/gcc/x86_64-redhat-linux/4.4.4/libgfortran.a \  #!!!!!!!!!!!!!USE MACHINES SPECIFIC LOCATIONS!!!!!!!!
	/usr/lib64/libdl.so										#!!!!!!!!!!!!!USE MACHINES SPECIFIC LOCATIONS!!!!!!!!
 
MUMPS_INCLUDE = -I$(MUMPS_INC)
 
endif
 
PARALLEL_LIB = 	
HPM_LIB =
 
 
MACHINE_NUMERICAL_LIBS  = /usr/lib/gcc/x86_64-redhat-linux/4.4.4/libgfortran.a \ #!!!!!!!!!!!!!USE MACHINES SPECIFIC LOCATIONS!!!!!!!!
-L/usr/lib64  -ldl -lm\															 #!!!!!!!!!!!!!USE MACHINES SPECIFIC LOCATIONS!!!!!!!!
	$(SUPERLU_LIBRARY) \
	$(UMFPACK_LIBRARY) \
	$(ARPACK_LIBRARY) \
	$(AMD_LIBRARY) \
	$(GRAPHIC_LIBRARY)\
	$(RELIABILITY_LIBRARY) \
	$(DISTRIBUTED_SUPERLU_LIBRARY) \
	$(METIS_LIBRARY) $(PARALLEL_LIB) $(DISTRIBUTED_SUPERLU_LIBRARY) $(PETSC_LIB) $(MUMPS_LIB) \
	$(SCALAP) $(parmetis_lib) $(metis_lib) $(SCALAP) \
	 -L/usr/lib64 -lssl -lcrypto\
 
 
MACHINE_SPECIFIC_LIBS = $(FE)/tcl/tclMain.o
 
# %---------------------------------------------------------%
# | SECTION 8: INCLUDE FILES                                |
# |                                                         |
# | The following macros specify include files needed for   |
# | compilation.                                            |
# %---------------------------------------------------------%
 
 
MACHINE_INCLUDES =	$(MPI_INC) \
			-I$(BERKLEYdir) \
			-I/usr/bin/mysql \ #!!!!!!!!!!!!!USE MACHINES SPECIFIC LOCATIONS!!!!!!!!
			-I$(UMFPACKdir) \
			-I$(SUPERLUdir) \
			-I$(SUPERLU_DISTdir) \
			 $(MUMPS_INCLUDE) \
			$(metis_inc)\
 			#$(PETSC_INC) 
			#-I$(OPEN_SSL_DIR)/include \		 
# this file contains all the OpenSees/SRC includes
include $(FE)/Makefile.incl
 
TCL_INCLUDES = -I$(TCLdir)/include
#-I$(FE)/tcl/include
#INCLUDES = $(TCL_INCLUDES) $(FE_INCLUDES) $(MACHINE_INCLUDES)
INCLUDES =  $(TCL_INCLUDES) $(MACHINE_INCLUDES) $(FE_INCLUDES)
 
####################################################################################################
 

MAKE

Inside OpenSees Folder type make. If it compiles successfully you will get a notification and OpenSeesMP will be created on bin folder.

Some Final Touch-ups

    • Copy init.tcl from $(TCLdir)/lib/tcl8.5/tcl.int to directory $(HOME)/lib/tcl8.5/
    • Entry of OpenSees to Environment $PATH , go to user home directory -
  • nano .bashrc
    

    Enter the following line at the end. Replace the "path_to_executable" with the path where OpenSeesMP is located.

    export PATH=/path_to_executable/OpenSeesMP:$PATH
    
    • Running examples –
  • cd to directory where examples are kept Use following command –

    mpirun –np 2 OpenSeesMP example.tcl
    


    Be sure to make mpich2 up and running before trying this.

     

     

    Created By : 	Khaled Mashfiq, MECRES, Sapienza University of Rome
    Courtesy :  	Prof. Rosario Gigliotti, Department of Structural Engineering, Sapienza University of Rome
                    Dr. Marco Faggella, Department of Structural Engineering, Sapienza University of Rome
                    Pietro Maioli, Department of Structural and Electrical Engineering, Sapienza University of Rome
                    SM Iftekharul Alam, Purdue University.
    扩展阅读