Polishing Diamonds in Java
Managing interface change in diamond hierarchies.
Inheriting Diamonds in Java
Java is an object-oriented language that supports single inheritance for classes. A class can inherit from at most one single parent class. Java also supports classes implementing multiple interfaces. Interfaces may extend multiple interfaces as well.
The following class diagram illustrates the two kinds of inheritance models supported in Java.
Before Java 8, methods on interfaces could only be abstract. It was the responsibility of classes to define the behavior of methods defined on interfaces.
Java 8 introduced an extremely powerful feature called default methods. A default method provides both the signature of a method, and a “default” implementation in a method body that can be used by classes that don’t override the behavior. The default methods feature can help make old interfaces new again.
Change happens. Unfortunately, change sometimes comes with a cost. Open Source Java projects have to be able to understand and respond quickly to change with the six month release cadence of OpenJDK feature releases. Open Source Java projects have a great feedback loop if they participate in the OpenJDK Quality Outreach Program. This program notifies members of the availability of early access releases of the OpenJDK that they can test their projects against.
The availability of early access releases of the OpenJDK has helped the Eclipse Collections open source project discover, understand, report, and address issues before new versions of the OpenJDK are released. In this blog, I will explain three times we had to make changes in Eclipse Collections after default methods were added to existing interfaces in the JDK.
Diamond Hierarchies
Inheriting from multiple interfaces can lead to the creation of diamond hierarchies. A diamond hierarchy gets its name from the shape of the hierarchy. Consider the following diagram showing 5 interfaces and a class in a diamond hierarchy relationship.
You may see different interface inheritance shapes in the wild. The shapes may not be diamonds. Sometimes you may encounter upside down trees. The diamond shapes themselves are not all that important. The most important characteristic that can lead to issues is that there exists a child interface or class at the bottom of a hierarchy that has multiple parents. Having multiple parent interfaces can lead to method signature collisions that may break compilation and potentially runtime behavior.
Diamond Hierarchies before Java 8
The primary interface inheritance problem before Java 8 occurred if two or more interfaces have the same method signatures with different return types.
Consider the following diamond hierarchy diagram that defines all abstract foo methods that are ultimately implemented by ClassD.
The interfaces A, B, C in this diagram all define foo methods that return different types that are covariant overrides of the parent interface Top. In order for ClassD to compile, it must override the foo method and return a type that is compatible with the foo methods from all three interfaces. The solution here is to override foo in ClassD and return ClassD. ClassD is a subtype of A, B, and C. ClassD is a covariant return type. Covariance is an important feature of Java return types that was added in Java 5.
Interfaces in the JDK were extremely stable before Java 8. We never saw any interface evolution issues in our diamond hierarchies in Eclipse Collections that integrated with Java types like Iterable, Collection, Map, List and Set. That changed slightly after Java 8.
Diamond Hierarchies after Java 8
Java 8 introduced a new feature called default methods. A default method allows a developer to add behavior to an existing interface without theoretically requiring any changes to classes that extend that interface. The default method feature has allowed the JDK to evolve interfaces that are decades old. We use default methods extensively in Eclipse Collections to reduce code duplication across abstract class hierarchies. There are a few gotchas to be aware of when using default methods, especially where there are diamond hierarchies that depend on interfaces that evolve over time.
Every default method that is added to decades-old interfaces like Iterable, Collection, Map, List, Set creates a possibility for unexpected method signature collisions to happen. Don’t be too alarmed for your applications. Most applications will probably never encounter the diamond hierarchy issues that Eclipse Collections and other libraries that provide Collection types that integrate with Java types may run into.
The following diagram shows the default methods that were added to the basic Collection interfaces in Java 8. Did any of these default methods break your applications when they arrived in Java 8? My guess would be, probably not.
The Map interface had the most notable evolution in Java 8 based on the number of default methods added. The methods that were added to Map were a much needed improvement for the Java community. While Eclipse Collections MutableMap has some functional overlap with methods in the Map interface, there were, fortunately, no method signature collisions with the new default methods that were added. Awesome!
The other Java Collection Framework interfaces had more modest additions, as most of the functionality was provided by the new Stream API. The spliterator, stream and parallelStream methods theoretically had the greatest possibility of collisions in the wild because the methods had zero arguments. In practice, I never saw or heard of any collisions that happened with these three methods. Awesome!
On the other hand, the forEach, removeIf, and replaceAll methods had method name collisions in frameworks like Eclipse Collections. Since the one argument types the methods required were all new in Java 8 (Consumer, Predicate, UnaryOperator), any collisions were simply considered overloads by the compiler. Awesome!
The method sort on List, which takes a decades-old interface named Comparator would result in collisions that created scratches in some diamonds in the wild. This was unfortunate, but sometimes the JDK needs to break some eggs in order to evolve and improve. List should have had a sort method from the beginning of its existence. Thankfully, it does now.
Scratching a Diamond by evolving interfaces
Every once in a while, a change can be made in an interface that requires “polishing” a diamond hierarchy. Interface changes may be out of your control if you have a relationship with an interface that is managed by another library or the JDK itself. Your only option may be to fix compilation failures once they are found and determine if there is also a binary incompatibility that may require a new release of your library or application in order for your clients to use a new version of the JDK or another library.
The following sections describe three different issues that may be encountered with evolving interfaces in diamond hierarchies. These are real examples that illustrate where Eclipse Collections had to address issues caused by the evolution of interfaces with default methods after Java 8.
Method collisions with different return types
The worst gotcha you can encounter with a diamond hierarchy is with methods signatures colliding with different return types. There is only one good solution to this problem. One of the methods must be renamed, and all client calls to the renamed method must be renamed as well.
When Java 8 was released, a default method sort was added to the java.util.List interface that had a void return type. Before we open sourced GS Collections, we had a sort method on the MutableList interface that returned MutableList. MutableList extends java.util.List, so our only option was to rename our method and change all of our client code to call the new method. Thankfully, all of the client code was in one company, so this was manageable with some explanation that compile errors would happen and there was a simple fix to change calls to sort that required a return type to sortThis.
This is what the sortThis method signatures looks like today on MutableList. These two methods were added as default methods on MutableList in the Eclipse Collections 10.0 release to reduce some code duplication.
default MutableList<T> sortThis(Comparator<? super T> comparator)
{
this.sort(comparator);
return this;
}
default MutableList<T> sortThis()
{
return this.sortThis(null);
}The method sortThis delegates to the method sort, which was added to the java.util.List interface as default method in Java 8. The sort method is then overridden in FastList, which implements MutableList.
@Override
public void sort(Comparator<? super T> comparator)
{
Arrays.sort(this.items, 0, this.size, comparator);
}If it isn’t obvious, the reason sortThis returns this, is so that it can be used fluently and directly as a return result in a method. This is an amazing convenience that often reduces lines of code when using sortThis.
The List change in Java 8 was described in this recent blog by Stuart Marks:
Following the advice in this blog, I am writing all of these experiences down for other maintainers to learn from. Thank you, Stuart!
Default method collisions can result in ambiguity
Another gotcha happens when two parent interfaces define default methods with the same exact signature. When this happens then a child interface must also override that default method as well to remove the ambiguity that arises. The compiler and runtime will not be able to determine which default method should be chosen as the implementation. This results in an ambiguity at compile time and runtime.
Eclipse Collections encountered this particular problem with JDK 15. Stuart Marks wrote a great blog describing the issue as well.
The only thing this blog is missing is a diagram to help visualize the issue. The following picture shows the colliding default methods in the hierarchy for CharAdapter.
The solution to this problem was to add an override of isEmpty in the CharAdapter class.
This hierarchy doesn’t have a full diamond shape. It does have the colliding method issue with isEmpty caused by extending multiple interfaces with the same method signatures for default methods.
Abstract and Default method collisions
In JDK 21, a new interface called SequencedCollection was added in between Collection and List that has methods like getFirst and getLast. These default methods collided with abstract methods with the same signature that were defined in RichIterable, OrderedIterable, and ListIterable in Eclipse Collections.
The following diagram shows the diamond hierarchy in Eclipse Collections and where the methods ultimately collide in the child interface MutableList.
Compilation Only Error
The collision between abstract getFirst methods in the Eclipse Collections types on the left and the default getFirst method in SequencedCollection on the right resulted in a compilation error in our JDK 21 EA builds for Eclipse Collections. All previous JDK versions compiled fine. What was unclear was whether this was only a compilation error, or if this would require a release of a new version of Eclipse Collections in order for the library to work with JDK 21 when it is released.
I checked to see if getFirst or getLast was used in either of the two OSS repos that I am a committer for that have an Eclipse Collections dependency and have a JDK 21 EA build. Both repos had tests that use getFirst. The code ran on JDK 21 EA using Eclipse Collections 11.1 without issue. The two repos are the Eclipse Collections Kata and BNY Mellon CodeKatas.
This should verify that the issue is compilation only. I found the section in the JLS that I believe covers this situation with colliding abstract and default methods in interface hierarchies.
This is a new kind of issue we hadn’t seen before that we can now be on the lookout for with future releases of the JDK.
Polishing this issue away
I am going to include code examples here which show how the problem manifest itself at compile time and the two potential solutions.
The following is a compilation error and the example code that creates the compilation issue with a diamond hierarchy.
java: types Diamond.SequencedCollection<E> and Diamond.ListIterable<T> are incompatible; interface Diamond.MutableList<T> inherits abstract and default for getFirst() from types Diamond.SequencedCollection and Diamond.ListIterable
public class Diamond
{
interface Iterable<E>
{
}
interface OrderedIterable<T> extends Iterable<T>
{
T getFirst();
}
interface ListIterable<T> extends OrderedIterable<T>
{
T getFirst();
}
interface Collection<E> extends Iterable<E>
{
}
interface SequencedCollection<E> extends Collection<E>
{
default E getFirst()
{
return null;
}
}
interface List<E> extends SequencedCollection<E>
{
}
interface MutableCollection<T> extends Collection<T>
{
}
interface MutableList<T> extends MutableCollection<T>, ListIterable<T>, List<T>
{
// This code doesn't compile and fails with error below:
// java: types Diamond.SequencedCollection<E> and
// Diamond.ListIterable<T> are incompatible;
// interface Diamond.MutableList<T> inherits abstract and default
// for getFirst() from types
// Diamond.SequencedCollection and Diamond.ListIterable
}
static class MyList<T> implements MutableList<T>, List<T>
{
public T getFirst()
{
return null;
}
}
public static void main(String[] args)
{
OrderedIterable<String> a = new MyList<>();
ListIterable<String> b = new MyList<>();
SequencedCollection<String> c = new MyList<>();
List<String> d = new MyList<>();
MutableList<String> e = new MyList<>();
MyList<String> f = new MyList<>();
System.out.println(a.getFirst());
System.out.println(b.getFirst());
System.out.println(c.getFirst());
System.out.println(d.getFirst());
System.out.println(e.getFirst());
}
}There are two possible solutions to solving this compilation issue. One solution is to add an abstract getFirst method in MutableList . The other solution is to add a default implementation for getFirst in MutableList.
The actual solution I used to solve the compilation issue in Eclipse Collections was to add default methods to MutableList for getFirst and getLast.
@Override
default T getFirst()
{
return this.isEmpty() ? null : this.get(0);
}
@Override
default T getLast()
{
return this.isEmpty() ? null : this.get(this.size() - 1);
}This getLast default method implementation would be suboptimal for LinkedList, but these two methods already have appropriate overrides in abstract and concrete classes. This solution's primary goal was to make the compiler happy.
Diamonds are forever so prepare to polish them
When diamond hierarchies or any multiple interface inheritance exist in a code base, care needs to be taken to upkeep them when interface evolution happens. Change does and will happen. I hope this blog demonstrates some useful real world example where rules in the Java Language Specification collide with real world libraries that are integrated with JDK interfaces.
I am quite happy as an Eclipse Collections maintainer how the ecosystem has evolved with Early Access versions of the JDK being provided with easy automation that we can leverage to use them. Getting a heads up months in advance on an upcoming change in the JDK is a huge improvement. Early warning capability for JDK and library developers is really amazing.
In case you want to learn more about the benefits of participating in the OpenJDK Quality Outreach Program, I will shamelessly plug my recent blog on the topic below.
Thank you for reading this blog! I hope you found the information and examples here useful. Enjoy!
I am the creator of and committer for the Eclipse Collections OSS project, which is managed at the Eclipse Foundation. Eclipse Collections is open for contributions.
