This article will discuss the use of the Pipes & Filters pattern.
First, we will analyze an example of a function, which we will rewrite later using the above-mentioned pattern. Changes in the code will occur gradually and each time we will create a workable version until we dwell on the solution using DI (in this Spring example).
Thus, we will create several solutions, providing the opportunity to use any.
In the end, we compare the initial and final implementations, look at examples of application in real projects and summarize.
Task
Suppose we have a bunch of clothes that we get from drying and which we now need to move to the closet. It turns out that the data (clothes) come from a separate service and the task is to provide this data to the client in the right form (in a closet from which he can get clothes).
In most cases, you cannot use the received data in the form in which it arrives to us. This data needs to be checked, transformed, sorted, etc.
Suppose a customer demands that clothes should be ironed if they are mint.
Then for the first time we create a Modifier , in which we prescribe the changes:
public class Modifier { public List<> modify(List<> ){ (); return ; } private void (List<> ) { .stream() .filter(::) .forEach(o -> {
At this stage, everything is simple and clear. Let's write a test that checks that all wrinkled clothes have been ironed.
But over time, new requirements appear and every time the functionality of the Modifier class expands:
- Do not put dirty laundry in the closet.
- Shirts, jackets and trousers should hang on the shoulders.
- Leaky socks need to be sewn up first
- ...
The sequence of changes is also important. For example, you can not first hang clothes on their shoulders, and then iron.
Thus, at some point, Modifier can take the following form:
public class Modifier { private static final Predicate<> ___ = ((Predicate<>).class::isInstance) .or(.class::isInstance) .or(.class::isInstance) ; public List<> modify(List<> ){ (); (); (); ();
Correspondingly, the tests have become more complicated, which now must at least check each step individually.
And when a new requirement arrives, looking at the code, we decide that the time has come for Refactoring.
Refactoring
The first thing that catches your eye is the frequent busting of all the clothes. So the first step, we move everything in one cycle, and also transfer the cleanliness check to the beginning of the cycle:
public class Modifier { private static final Predicate<> ___ = ((Predicate<>).class::isInstance) .or(.class::isInstance) .or(.class::isInstance) ; public List<> modify(List<> ){ List<> result = new ArrayList<>(); for(var o : ){ if(o.()){ continue; } result.add(o); (o); (o); (o);
Now, the processing time for clothes is reduced, but the code is still too long for one class and for the body of the cycle. Let's try to shorten the body of the cycle first.
After checking for cleanliness, you can make all the calls in a separate modify( ) method modify( ) :
public List<> modify(List<> ){ List<> result = new ArrayList<>(); for(var o : ){ if(o.()){ continue; } result.add(o); modify(o); } return result; } private void modify( o) { (o); (o); (o);
You can combine all the calls into one Consumer :
private Consumer<> modification = ((Consumer<>) this::) .andThen(this::) .andThen(this::);
Blunt: peek
I used peek for short. Sonar will say that such a code should not be done, because Javadoc tells peek that the method exists primarily for debug. But if you rewrite it on map: .map (o -> {modification.accept (o); return o;}), then IDEA will say that it is better to use peek
Stumbling: Consumer
An example with Consumer (and subsequent with Function) is given to show the capabilities of the language.
Now the body of the cycle has become shorter, but so far the class itself is still too large and contains too much information (knowledge of all steps).
Let's try to solve this problem using already established programming patterns. In this case, we will use Pipes & Filters .
Pipes & filters
The channel and filter template describes an approach in which incoming data goes through several processing steps.
Let's try to apply this approach to our code.
Step 1
In fact, our code is already close to this pattern. The data obtained go through several independent steps. So far, each method is a filter, and modify itself describes the channel, first filtering out all dirty clothes.
Now we will transfer each step to a separate class and see what we get:
public class Modifier { private final ; private final ; private final ;
Thus, we placed the code in separate classes, simplifying the tests for individual transformations (and creating the possibility of reusing steps). The order of the calls determines the sequence of steps.
But the class itself still knows all the individual steps, controls the order and thus has a huge list of dependencies. In addition to adding a new step, we will be forced to not only write a new class, but also add it to Modfier .
Step 2
Simplify the code using Spring.
First, create an interface for each individual step:
interface Modification { void modify( ); }
Modifier itself will now be much shorter:
public class Modifier { private final List<Modification> steps; @Autowired public Modifier(List<Modification> steps) { this.steps = steps; } public List<> modify(List<> ) { return .stream() .filter(o -> !o.()) .peek(o -> { steps.forEach(m -> m.modify(o)); }) .collect(Collectors.toList()); } }
Now, to add a new step, you just need to write a new class that implements the Modification interface and put @Component above it. Spring will find it and add it to the list.
Modifer itself Modifer not know anything about the individual steps, which creates a “weak connection” between the components.
The only difficulty is to set the sequence. To do this, Spring has an @Order annotation into which you can pass an int value. The list is sorted in ascending order.
Thus, it may happen that by adding a new step in the middle of the list, you will have to change the sorting values for existing steps.
Spring could have been dispensed with if all known implementations were manually passed to the Modifier constructor. This will help solve the sorting problem, but again complicate the addition of new steps.
Step 3
Now we pass the test for cleanliness in a separate step. To do this, we rewrite our interface so that it always returns a value:
interface Modification { modify( ); }
Check for cleanliness:
@Component @Order(Ordered.HIGHEST_PRECEDENCE) class CleanFilter implements Modification { modify( ) { if(.()){ return null; } return ; } }
Modifier.modify :
public List<> modify(List<> ) { return .stream() .map(o -> { var modified = o; for(var step : steps){ modified = step.modify(o); if(modified == null){ return null; } } return modified; }) .filter(Objects::nonNull) .collect(Collectors.toList()); }
In this version, Modifier does not have any data information. He simply passes them on to every known step and collects the results.
If one of the steps returns null, then processing for this garment is interrupted.
A similar principle is used in Spring for HandlerInterceptors. Before and after the controller call, all appropriate Interceptors for this URL are called. At the same time, it returns true or false in the preHandle method to indicate whether processing and calling subsequent Interceptors can continue
Step n
The next step is to add the matches method to the Modification interface, in which the steps to a separate attribute of the clothes are checked:
interface Modification { modify( ); default matches( ) {return true;} }
Due to this, you can slightly simplify the logic in modify methods by moving the checks for classes and properties into a separate method.
A similar approach is used in the Spring (Request) Filter, but the main difference is that each Filter is a wrapper around the next and explicitly calls FilterChain.doFilter to continue processing.
Total
The end result is very different from the initial version. Comparing them, we can draw the following conclusions:
- The implementation based on Pipes & Filters simplifies the
Modifier class itself. - Better distributed responsibilities and “weak” connections between components.
- Easier to test individual steps.
- Easier to add and remove steps.
- A little harder to test a whole chain of filters. We need IntegrationTests already.
- More classes
Ultimately, a more convenient and flexible option than the original.
In addition, you can simply parallelize data processing using the same parallelStream.
What this example does not solve
- The description of the pattern says that individual filters can be reused by creating another filter chain (channel).
- On the one hand, this is easy to do using
@Qualifier . - On the other hand, setting a different order with
@Order will fail.
- For more complex examples, you will have to use several chains, use nested chains, and still change the existing implementation.
- So for example, the task: "for each sock, look for a pair and put them into one instance of <? Extends Clothing>" will not fit well into the described implementation, because Now, for each toe, you have to sort through all the linen and change the initial data list.
- To solve this, you can write a new interface that accepts and returns a List <Clothing> and transfers it to a new chain. But you need to be careful with the sequence of calls of the chains themselves, if the socks can be sewn only by hotel.
Thanks for attention