Some of these practices might be based on wrong assumptions and I'm not aware of it, so I would appreciate any feedback.
-
avoiding some dependency conflicts:
- install sbt-explicit-dependencies globally in your
~/.sbt/{0.13,1.0}/plugins/plugins.sbt - run
undeclaredCompileDependenciesand make the obvious missing dependencies explicit by adding them tolibraryDependenciesof each sub-project - (optionally) run
unusedCompileDependenciesand remove some obvious unused libraries. This has false positives, so; reload; Test/compileafter each change and ultimately run all tests to see that it didn't break anything - (optionally) add
undeclaredCompileDependenciesTestto the CI pipeline, so that it will fail if you have some undeclared dependencies
- install sbt-explicit-dependencies globally in your
-
keeping dependencies up to date and resolving conflicts:
- install sbt-updates globally in your
~/.sbt/{0.13,1.0}/plugins/plugins.sbt - run
dependencyUpdatesand bump all non-major versions. Major versions updates should be done one by one with careand loveand testing. - include all explicit
libraryDependenciesindependencyOverridesto force their versions. This is supposed to have the same effect as applyingforce()on alllibraryDependencies, but isn't ivy-specific. The point is to prevent conflict manager choosing automatically some version required by a transitive dependency instead of the one you wrote explicitly. This is done inproject/Dependencies.scala - it might become difficult to maintain versions in
libraryDependenciesanddependencyOverridesin sync, so a common practice is to define values for versions of each dependency, put them inproject/Versions.scalaand use throughoutbuild.sbt. This is also convenient for libraries that are split in multiple artifacts which have to have the same version.
- install sbt-updates globally in your
-
using sbt-assembly:
- try running
assemblyfor each project (starting from the independent ones) and see if there are any merge conflicts - if there are two different libraries that contain conflicting class files (same path, different content), use shading to rename one of them:
- don't use
.inAllbecause it will rename classes in both of the libraries and it will be the same situation. Instead use.inLibraryor.inProject - you might need to use
.inAllfor something else, e.g. you have a predefined runtime classpath with some outdated libraries and want to use newer versions, but avoid conflicts with that external classpath
- don't use
- avoid using
excludeorexcludeDependenciesbecause you may throw away some library which is needed by one of the transitive dependencies and it will fail in runtime withMethodNotFoundExceptionor something like that - avoid overriding merge strategy on class files (using
first/last/discardstrategies), because it's the same as excluding some classes. Use merge strategy overrides only for some trivial conflicts or non-class files, e.g. to merge two.propertiesfiles withconcatorfilterDistinctLinesstrategy
- try running
-
resolving more conflicts:
- run
evictedfor each project and inspect the list of automatically resolved conflicts - try to minimize the number of lines marked as
[warn], those are conflicts with potentially binary incompatible versions - if some of the libraries introducing the conflict are yours (company-owned), go and update its dependencies to solve the conflict. Unfortunately, it's more often the other way around: your libraries are more up to date than some external ones that you don't have access to. Anyway, consider updating those external libraries or contribute to them if they are opensource.
- use sbt-dependency-graph installed globally to untangle the dependencies and understand the origin of the conflicts. The
whatDependsOntask is very useful for that.
- run
-
Rinse and repeat. I numbered these steps because IMO it's better to do them in this order, but after each step it might be useful to go through the previous steps again.
Some useful links:
-
"What are your tips/tricks/tools for dealing with dependency hell?"
- https://github.com/cb372/sbt-explicit-dependencies
- good point (human factor):
Do a better job of articulating WHY certain dependencies are used rather than dumping everything in
commonand then wondering why SBT is so complex.
-
advices from Eugene Yokota:
- avoid -SNAPSHOT outside of local testing
- avoid version range
-
evictedfor eviction report - sbt -no-colors update > update.txt
- jrudolph/sbt-dependency-graph
- rtimush/sbt-updates
-
semantics of dependency resolvers by Eugene Yokota
the exact rules governing how the user-specified constraints are interpreted vary from one dependency resolver to another. I call these rules the semantics of the dependency resolvers.
-
sbt docs:
- library management
- dependency management flow
- strict conflict manager
- I decided not to use it, because it works only in the cases when you have a limited number of dependencies, most of them are explicit and controlled by you (i.e. minimum external dependencies, or just NIH syndrome). I can expand my arguments here if needed.
This is an outstanding guide.