Skip to content

Instantly share code, notes, and snippets.

@finsberg
Created March 8, 2024 12:45
Show Gist options
  • Select an option

  • Save finsberg/da883a73ad6ac153fe77177a6cb08ea2 to your computer and use it in GitHub Desktop.

Select an option

Save finsberg/da883a73ad6ac153fe77177a6cb08ea2 to your computer and use it in GitHub Desktop.
Running nonlinear hyperelasticity with double and single precision in dolfinx based on https://jsdokken.com/dolfinx-tutorial/chapter2/hyperelasticity.html
from ghcr.io/fenics/test-env:current-mpich
ENV PETSC_ARCH=linux-gnu-real32-32
ARG UFL_BRANCH=main
ARG FFCX_BRANCH=main
ARG BUILD_MODE=Release
RUN python3 -m pip install pybind11
RUN git clone --branch=main --single-branch https://github.com/FEniCS/dolfinx.git
RUN git clone --branch=main --single-branch https://github.com/FEniCS/basix.git
RUN cmake -G Ninja -B build-basix -DCMAKE_BUILD_TYPE="Release" -S ./basix/cpp/
RUN cmake --build build-basix --parallel 2
RUN cmake --install build-basix
RUN python3 -m pip -v install --check-build-dependencies --config-settings=build-dir="build-basix-python" --config-settings=cmake.build-type=${BUILD_MODE} --config-settings=install.strip=false --no-build-isolation ./basix/python
RUN python3 -m pip install git+https://github.com/FEniCS/ufl.git@${UFL_BRANCH}
RUN python3 -m pip install git+https://github.com/FEniCS/ffcx.git@${FFCX_BRANCH}
RUN git clone --branch=master --single-branch https://github.com/ornladios/ADIOS2.git adios2
RUN cmake -G Ninja -DADIOS2_USE_HDF5=on -DCMAKE_INSTALL_PYTHONDIR=/usr/local/lib/ -DADIOS2_USE_Fortran=off -DBUILD_TESTING=off -DADIOS2_BUILD_EXAMPLES=off -DADIOS2_USE_ZeroMQ=off -B build-dir-adios2 -S ./adios2
RUN cmake -G Ninja -B build-dir-adios2 -DCMAKE_BUILD_TYPE="Release" -S ./adios2/
RUN cmake --build build-dir-adios2 --parallel 2
RUN cmake --install build-dir-adios2
ENV PETSC_DIR=/usr/local/petsc
RUN PETSC_DIR=$PETSC_DIR PETSC_ARCH=${PETSC_ARCH} cmake -G Ninja -DCMAKE_BUILD_TYPE=${BUILD_MODE} -B build-dolfinx -S ./dolfinx/cpp/
RUN cmake --build build-dolfinx
RUN cmake --install build-dolfinx
RUN PETSC_DIR=${PETSC_DIR} PETSC_ARCH=${PETSC_ARCH} python3 -m pip -v install --check-build-dependencies --no-build-isolation ./dolfinx/python/
from dolfinx import log, default_scalar_type
from dolfinx.fem.petsc import NonlinearProblem
from dolfinx.nls.petsc import NewtonSolver
from dolfinx import nls
import numpy as np
import ufl
import dolfinx
from mpi4py import MPI
from dolfinx import fem, mesh, plot
L = 20.0
domain = mesh.create_box(MPI.COMM_WORLD, [[0.0, 0.0, 0.0], [L, 1, 1]], [20, 5, 5], mesh.CellType.hexahedron, dtype=dolfinx.default_real_type)
V = dolfinx.fem.functionspace(domain, ("Lagrange", 2, (3,)))
# +
def left(x):
return np.isclose(x[0], 0)
def right(x):
return np.isclose(x[0], L)
fdim = domain.topology.dim - 1
left_facets = mesh.locate_entities_boundary(domain, fdim, left)
right_facets = mesh.locate_entities_boundary(domain, fdim, right)
# -
# Next, we create a marker based on these two functions
# Concatenate and sort the arrays based on facet indices. Left facets marked with 1, right facets with two
marked_facets = np.hstack([left_facets, right_facets])
marked_values = np.hstack([np.full_like(left_facets, 1), np.full_like(right_facets, 2)])
sorted_facets = np.argsort(marked_facets)
facet_tag = mesh.meshtags(domain, fdim, marked_facets[sorted_facets], marked_values[sorted_facets])
# We then create a function for supplying the boundary condition on the left side, which is fixed.
u_bc = np.array((0,) * domain.geometry.dim, dtype=default_scalar_type)
# To apply the boundary condition, we identity the dofs located on the facets marked by the `MeshTag`.
left_dofs = fem.locate_dofs_topological(V, facet_tag.dim, facet_tag.find(1))
bcs = [fem.dirichletbc(u_bc, left_dofs, V)]
# Next, we define the body force on the reference configuration (`B`), and nominal (first Piola-Kirchhoff) traction (`T`).
B = fem.Constant(domain, default_scalar_type((0, 0, 0)))
T = fem.Constant(domain, default_scalar_type((0, 0, 0)))
# Define the test and solution functions on the space $V$
v = ufl.TestFunction(V)
u = fem.Function(V)
# +
# Spatial dimension
d = len(u)
# Identity tensor
I = ufl.variable(ufl.Identity(d))
# Deformation gradient
F = ufl.variable(I + ufl.grad(u))
# Right Cauchy-Green tensor
C = ufl.variable(F.T * F)
# Invariants of deformation tensors
Ic = ufl.variable(ufl.tr(C))
J = ufl.variable(ufl.det(F))
# -
# Define the elasticity model via a stored strain energy density function $\psi$, and create the expression for the first Piola-Kirchhoff stress:
# Elasticity parameters
E = default_scalar_type(1.0e4)
nu = default_scalar_type(0.3)
mu = fem.Constant(domain, default_scalar_type(E / (2 * (1 + nu))))
lmbda = fem.Constant(domain, default_scalar_type(E * nu / ((1 + nu) * (1 - 2 * nu))))
# Stored strain energy density (compressible neo-Hookean model)
psi = (mu / 2) * (Ic - 3) - mu * ufl.ln(J) + (lmbda / 2) * (ufl.ln(J))**2
# Stress
# Hyper-elasticity
P = ufl.diff(psi, F)
metadata = {"quadrature_degree": 6}
ds = ufl.Measure('ds', domain=domain, subdomain_data=facet_tag, metadata=metadata)
dx = ufl.Measure("dx", domain=domain)#, metadata=metadata)
# Define form F (we want to find u such that F(u) = 0)
F = ufl.inner(ufl.grad(v), P) * dx - ufl.inner(v, B) * dx - ufl.inner(v, T) * ds(2)
# As the varitional form is non-linear and written on residual form, we use the non-linear problem class from DOLFINx to set up required structures to use a Newton solver.
problem = NonlinearProblem(F, u, bcs)
# and then create and customize the Newton solver
# +
solver = NewtonSolver(domain.comm, problem)
# Set Newton solver options
solver.atol = 1e-5
solver.rtol = 1e-5
solver.krylov_solver.setType("preonly")
solver.krylov_solver.getPC().setType("lu")
solver.krylov_solver.getPC().setFactorSolverType("mumps")
solver.max_it = 25
solver.convergence_criterion = "residual"
log.set_log_level(log.LogLevel.INFO)
tval0 = -1.5
for n in range(1, 2):
T.value[2] = n * tval0
num_its, converged = solver.solve(u)
assert (converged)
u.x.scatter_forward()
print(f"Time step {n}, Number of iterations {num_its}, Load {T.value}")
@finsberg
Copy link
Author

finsberg commented Mar 8, 2024

To run with fp64 you can use the the docker image ghcr.io/fenics/dolfinx/dolfinx:nightly, e.g

docker run --rm -v $PWD:/shared -w /shared -it ghcr.io/fenics/dolfinx/dolfinx:nightly python3 hyperelasticity.py

which gives the following output

2024-03-08 12:04:40.997 (   7.201s) [main            ]       NewtonSolver.cpp:36    INFO| Newton iteration 0: r (abs) = 0.163333 (tol = 1e-05) r (rel) = inf(tol = 1e-05)
2024-03-08 12:05:03.934 (  30.138s) [main            ]              petsc.cpp:698   INFO| PETSc Krylov solver starting to solve system.
2024-03-08 12:05:06.985 (  33.189s) [main            ]       NewtonSolver.cpp:36    INFO| Newton iteration 1: r (abs) = 1330.75 (tol = 1e-05) r (rel) = 8.03268(tol = 1e-05)
2024-03-08 12:05:30.141 (  56.345s) [main            ]              petsc.cpp:698   INFO| PETSc Krylov solver starting to solve system.
2024-03-08 12:05:33.087 (  59.291s) [main            ]       NewtonSolver.cpp:36    INFO| Newton iteration 2: r (abs) = 73.9037 (tol = 1e-05) r (rel) = 0.446096(tol = 1e-05)
2024-03-08 12:05:56.093 (  82.298s) [main            ]              petsc.cpp:698   INFO| PETSc Krylov solver starting to solve system.
2024-03-08 12:05:59.048 (  85.252s) [main            ]       NewtonSolver.cpp:36    INFO| Newton iteration 3: r (abs) = 0.471432 (tol = 1e-05) r (rel) = 0.00284565(tol = 1e-05)
2024-03-08 12:06:22.142 ( 108.346s) [main            ]              petsc.cpp:698   INFO| PETSc Krylov solver starting to solve system.
2024-03-08 12:06:25.113 ( 111.317s) [main            ]       NewtonSolver.cpp:36    INFO| Newton iteration 4: r (abs) = 0.904556 (tol = 1e-05) r (rel) = 0.00546007(tol = 1e-05)
2024-03-08 12:06:48.156 ( 134.359s) [main            ]              petsc.cpp:698   INFO| PETSc Krylov solver starting to solve system.
2024-03-08 12:06:51.148 ( 137.351s) [main            ]       NewtonSolver.cpp:36    INFO| Newton iteration 5: r (abs) = 0.00101159 (tol = 1e-05) r (rel) = 6.10614e-06(tol = 1e-05)
2024-03-08 12:06:51.148 ( 137.351s) [main            ]       NewtonSolver.cpp:250   INFO| Newton solver finished in 5 iterations and 5 linear solver iterations.

To run the single precision you first need to build dolfinx with single precision (using the given Dockerfile), i.e

docker build -t dolfinx-fp32 -f Dockerfile_fp32 .

and then run the demo in the container

docker run --rm -v $PWD:/shared -w /shared -it dolfinx-fp32 python3 hyperelasticity.py

which gives the following output

2024-03-08 12:03:38.772 (   6.019s) [main            ]       NewtonSolver.cpp:36    INFO| Newton iteration 0: r (abs) = 0.163333 (tol = 1e-05) r (rel) = inf(tol = 1e-05)
2024-03-08 12:04:01.106 (  28.353s) [main            ]              petsc.cpp:698   INFO| PETSc Krylov solver starting to solve system.
2024-03-08 12:04:03.122 (  30.369s) [main            ]       NewtonSolver.cpp:36    INFO| Newton iteration 1: r (abs) = 0.941543 (tol = 1e-05) r (rel) = 0.793011(tol = 1e-05)
2024-03-08 12:04:25.691 (  52.938s) [main            ]              petsc.cpp:698   INFO| PETSc Krylov solver starting to solve system.
2024-03-08 12:04:27.622 (  54.869s) [main            ]       NewtonSolver.cpp:36    INFO| Newton iteration 2: r (abs) = 19.4286 (tol = 1e-05) r (rel) = 16.3637(tol = 1e-05)
2024-03-08 12:04:50.543 (  77.790s) [main            ]              petsc.cpp:698   INFO| PETSc Krylov solver starting to solve system.
2024-03-08 12:04:52.548 (  79.795s) [main            ]       NewtonSolver.cpp:36    INFO| Newton iteration 3: r (abs) = 428.533 (tol = 1e-05) r (rel) = 360.93(tol = 1e-05)
2024-03-08 12:05:15.603 ( 102.850s) [main            ]              petsc.cpp:698   INFO| PETSc Krylov solver starting to solve system.
2024-03-08 12:05:17.606 ( 104.852s) [main            ]       NewtonSolver.cpp:36    INFO| Newton iteration 4: r (abs) = 138.953 (tol = 1e-05) r (rel) = 117.033(tol = 1e-05)
2024-03-08 12:05:40.700 ( 127.947s) [main            ]              petsc.cpp:698   INFO| PETSc Krylov solver starting to solve system.
2024-03-08 12:05:42.667 ( 129.914s) [main            ]       NewtonSolver.cpp:36    INFO| Newton iteration 5: r (abs) = 8955.04 (tol = 1e-05) r (rel) = 7542.35(tol = 1e-05)
2024-03-08 12:06:05.755 ( 153.001s) [main            ]              petsc.cpp:698   INFO| PETSc Krylov solver starting to solve system.
2024-03-08 12:06:07.719 ( 154.965s) [main            ]       NewtonSolver.cpp:36    INFO| Newton iteration 6: r (abs) = nan (tol = 1e-05) r (rel) = nan(tol = 1e-05)
2024-03-08 12:06:30.641 ( 177.887s) [main            ]              petsc.cpp:698   INFO| PETSc Krylov solver starting to solve system.
2024-03-08 12:06:32.341 ( 179.587s) [main            ]       NewtonSolver.cpp:36    INFO| Newton iteration 7: r (abs) = nan (tol = 1e-05) r (rel) = nan(tol = 1e-05)
2024-03-08 12:06:55.214 ( 202.460s) [main            ]              petsc.cpp:698   INFO| PETSc Krylov solver starting to solve system.
2024-03-08 12:06:56.482 ( 203.728s) [main            ]       NewtonSolver.cpp:36    INFO| Newton iteration 8: r (abs) = nan (tol = 1e-05) r (rel) = nan(tol = 1e-05)
2024-03-08 12:07:18.918 ( 226.164s) [main            ]              petsc.cpp:698   INFO| PETSc Krylov solver starting to solve system.
2024-03-08 12:07:20.195 ( 227.441s) [main            ]       NewtonSolver.cpp:36    INFO| Newton iteration 9: r (abs) = nan (tol = 1e-05) r (rel) = nan(tol = 1e-05)
2024-03-08 12:07:42.507 ( 249.753s) [main            ]              petsc.cpp:698   INFO| PETSc Krylov solver starting to solve system.
2024-03-08 12:07:43.752 ( 250.998s) [main            ]       NewtonSolver.cpp:36    INFO| Newton iteration 10: r (abs) = nan (tol = 1e-05) r (rel) = nan(tol = 1e-05)
2024-03-08 12:08:06.358 ( 273.603s) [main            ]              petsc.cpp:698   INFO| PETSc Krylov solver starting to solve system.
2024-03-08 12:08:07.607 ( 274.852s) [main            ]       NewtonSolver.cpp:36    INFO| Newton iteration 11: r (abs) = nan (tol = 1e-05) r (rel) = nan(tol = 1e-05)
2024-03-08 12:08:30.144 ( 297.389s) [main            ]              petsc.cpp:698   INFO| PETSc Krylov solver starting to solve system.
2024-03-08 12:08:31.406 ( 298.650s) [main            ]       NewtonSolver.cpp:36    INFO| Newton iteration 12: r (abs) = nan (tol = 1e-05) r (rel) = nan(tol = 1e-05)
2024-03-08 12:08:53.789 ( 321.034s) [main            ]              petsc.cpp:698   INFO| PETSc Krylov solver starting to solve system.
2024-03-08 12:08:55.055 ( 322.300s) [main            ]       NewtonSolver.cpp:36    INFO| Newton iteration 13: r (abs) = nan (tol = 1e-05) r (rel) = nan(tol = 1e-05)
2024-03-08 12:09:17.070 ( 344.315s) [main            ]              petsc.cpp:698   INFO| PETSc Krylov solver starting to solve system.
2024-03-08 12:09:18.340 ( 345.584s) [main            ]       NewtonSolver.cpp:36    INFO| Newton iteration 14: r (abs) = nan (tol = 1e-05) r (rel) = nan(tol = 1e-05)
2024-03-08 12:09:40.309 ( 367.553s) [main            ]              petsc.cpp:698   INFO| PETSc Krylov solver starting to solve system.
2024-03-08 12:09:41.550 ( 368.795s) [main            ]       NewtonSolver.cpp:36    INFO| Newton iteration 15: r (abs) = nan (tol = 1e-05) r (rel) = nan(tol = 1e-05)
2024-03-08 12:10:03.551 ( 390.795s) [main            ]              petsc.cpp:698   INFO| PETSc Krylov solver starting to solve system.
2024-03-08 12:10:04.809 ( 392.053s) [main            ]       NewtonSolver.cpp:36    INFO| Newton iteration 16: r (abs) = nan (tol = 1e-05) r (rel) = nan(tol = 1e-05)
2024-03-08 12:10:26.833 ( 414.077s) [main            ]              petsc.cpp:698   INFO| PETSc Krylov solver starting to solve system.
2024-03-08 12:10:28.054 ( 415.298s) [main            ]       NewtonSolver.cpp:36    INFO| Newton iteration 17: r (abs) = nan (tol = 1e-05) r (rel) = nan(tol = 1e-05)
2024-03-08 12:10:50.033 ( 437.278s) [main            ]              petsc.cpp:698   INFO| PETSc Krylov solver starting to solve system.
2024-03-08 12:10:51.276 ( 438.520s) [main            ]       NewtonSolver.cpp:36    INFO| Newton iteration 18: r (abs) = nan (tol = 1e-05) r (rel) = nan(tol = 1e-05)
2024-03-08 12:11:13.218 ( 460.462s) [main            ]              petsc.cpp:698   INFO| PETSc Krylov solver starting to solve system.
2024-03-08 12:11:14.468 ( 461.712s) [main            ]       NewtonSolver.cpp:36    INFO| Newton iteration 19: r (abs) = nan (tol = 1e-05) r (rel) = nan(tol = 1e-05)
2024-03-08 12:11:36.789 ( 484.032s) [main            ]              petsc.cpp:698   INFO| PETSc Krylov solver starting to solve system.
2024-03-08 12:11:38.018 ( 485.261s) [main            ]       NewtonSolver.cpp:36    INFO| Newton iteration 20: r (abs) = nan (tol = 1e-05) r (rel) = nan(tol = 1e-05)
2024-03-08 12:12:00.234 ( 507.477s) [main            ]              petsc.cpp:698   INFO| PETSc Krylov solver starting to solve system.
2024-03-08 12:12:01.503 ( 508.747s) [main            ]       NewtonSolver.cpp:36    INFO| Newton iteration 21: r (abs) = nan (tol = 1e-05) r (rel) = nan(tol = 1e-05)
2024-03-08 12:12:23.879 ( 531.123s) [main            ]              petsc.cpp:698   INFO| PETSc Krylov solver starting to solve system.
2024-03-08 12:12:25.137 ( 532.381s) [main            ]       NewtonSolver.cpp:36    INFO| Newton iteration 22: r (abs) = nan (tol = 1e-05) r (rel) = nan(tol = 1e-05)
2024-03-08 12:12:47.123 ( 554.366s) [main            ]              petsc.cpp:698   INFO| PETSc Krylov solver starting to solve system.
2024-03-08 12:12:48.422 ( 555.665s) [main            ]       NewtonSolver.cpp:36    INFO| Newton iteration 23: r (abs) = nan (tol = 1e-05) r (rel) = nan(tol = 1e-05)
2024-03-08 12:13:10.456 ( 577.698s) [main            ]              petsc.cpp:698   INFO| PETSc Krylov solver starting to solve system.
2024-03-08 12:13:11.677 ( 578.920s) [main            ]       NewtonSolver.cpp:36    INFO| Newton iteration 24: r (abs) = nan (tol = 1e-05) r (rel) = nan(tol = 1e-05)
2024-03-08 12:13:33.664 ( 600.906s) [main            ]              petsc.cpp:698   INFO| PETSc Krylov solver starting to solve system.
2024-03-08 12:13:34.909 ( 602.151s) [main            ]       NewtonSolver.cpp:36    INFO| Newton iteration 25: r (abs) = nan (tol = 1e-05) r (rel) = nan(tol = 1e-05)
Traceback (most recent call last):
  File "/shared/demo/demo_hyper.py", line 199, in <module>
    num_its, converged = solver.solve(u)
  File "/usr/local/lib/python3.10/dist-packages/dolfinx/nls/petsc.py", line 47, in solve
    n, converged = super().solve(u.vector)
RuntimeError: Newton solver did not converge because maximum number of iterations reached
2024-03-08 12:13:34.987 ( 602.229s) [main            ]             loguru.cpp:526   INFO| atexit

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment