C# Design Patterns - Iterator - The Pattern
If you've written any C# code that works with collections (almost all code), then you've almost certainly used the iterator pattern. This is the first blog post in a two-part series on iterators; in this post I'll explain the iterator pattern and show how you can implement it from scratch. In the next post I'll show how the iterator pattern is built into C# and how pivotal it is to much of the code we write.
A Scenario
I'm going to set out a (fictitious) scenario to help explain why the iterator pattern is useful and how it works.
Let's imagine you're working for a company developing accountancy software. The company already has an implementation of the SalesAccount and ExpenseAccount types, each of which store Transaction objects. The SalesAccount is intended to store sales transactions, that is, transactions where money has been paid in; the ExpenseAccount is intended to store expense transactions, i.e. where money has been paid out.
These two accounts types were developed by different teams in the company, each of which chose to use different C# collections for storing the Transaction objects internally; SalesAccount uses a list - List<Transaction>, and ExpenseAccount uses an array - Transaction[]. They both use the same implementation of the Transaction type. The current type definitions are shown below.
public record Transaction (string Name, float Amount, float TaxRate, bool IsReconciled);
public class SalesAccount
{
public List<Transaction> Transactions { get; }
public SalesAccount()
{
Transactions = new List<Transaction>();
AddTransaction(
name: "Stark Industries",
amount: 200000000,
taxRate: 0,
isReconciled: false
);
AddTransaction(
name: "Wayne Enterprises",
amount: 5500000,
taxRate: 10,
isReconciled: true
);
AddTransaction(
name: "Oscorp",
amount: 100000,
taxRate: 20,
isReconciled: false
);
}
public void AddTransaction(string name, float amount, float taxRate, bool isReconciled)
{
Transaction transaction = new Transaction(name, amount, taxRate, isReconciled);
this.Transactions.Add(transaction);
}
}
public class ExpenseAccount
{
private static int maxItems = 3;
private int numberOfItems = 0;
public Transaction[] Transactions { get; }
public ExpenseAccount()
{
Transactions = new Transaction[maxItems];
AddTransaction(
name: "Gotham City Iron Works",
amount: -1500000,
taxRate: 10,
isReconciled: true
);
AddTransaction(
name: "Super Electronics",
amount: -100000,
taxRate: 20,
isReconciled: false
);
AddTransaction(
name: "Wakanda Vibranium Corporation",
amount: -100000000,
taxRate: 0,
isReconciled: true
);
}
public void AddTransaction(string name, float amount, float taxRate, bool isReconciled)
{
Transaction transaction = new (name, amount, taxRate, isReconciled);
if (numberOfItems >= maxItems)
{
Console.WriteLine("Account is full! Can't add transaction to account");
}
else
{
Transactions[numberOfItems] = transaction;
numberOfItems += 1;
}
}
}
The problem
The Accountant type, which is a client of the two account types, is currently under development. Its requirements include being able to print all transactions, print all transactions that are reconciled, and calculate the total profit. The current account types have much code depending on them, meaning modification is out of the question.
Clearly, these requirements can be implemented by looping over the Transaction items in each of the accounts and doing something with each of them, e.g. printing their details to the console, in the case of the "print all transactions" requirement. So, let's see how the PrintTransactions method on Accountant would look.
public class Accountant
{
private ExpenseAccount expenseAccount;
private SalesAccount salesAccount;
public Accountant (ExpenseAccount expenseAccount, SalesAccount salesAccount)
{
this.expenseAccount = expenseAccount;
this.salesAccount = salesAccount;
}
public void PrintTransactions()
{
List<Transaction> salesTransactions = salesAccount.Transactions;
Transaction[] expenseTransactions = expenseAccount.Transactions;
for (int i = 0; i < salesTransactions.Count; i++)
{
Transaction transaction = salesTransactions[i];
Console.WriteLine($"{transaction.Name}\n{transaction.Amount}\n{transaction.TaxRate}\n{transaction.IsReconciled}");
Console.WriteLine();
}
for (int i = 0; i < expenseTransactions.Length; i++)
{
Transaction transaction = expenseTransactions[i];
Console.WriteLine($"{transaction.Name}\n{transaction.Amount}\n{transaction.TaxRate}\n{transaction.IsReconciled}");
Console.WriteLine();
}
}
}
The Accountant type takes an ExpenseAccount and a SalesAccount as arguments to its constructor, the PrintTransactions method then simply loops through each of the collections from the the two account objects and prints details of their Transaction items. As shown by the code below and the corresponding output as a result of running it.
Accountant accountant = new(
new ExpenseAccount(),
new SalesAccount());
accountant.PrintTransactions();
Why is this code bad?
The method contains two loops for iterating over the collections, and they need to work slightly differently because one iterates over an array, and the other iterates over a ListAccountant type - they're each going to require multiple loops to iterate over the items in the various account objects, therefore we're going to have a lot of duplicate code in a single class.
The Accountant needs to know how each of the accounts represents its collection of Transactions internally, this violates encapsulation. All the Accountant is required to do in this instance is loop through the Transaction items in each of the accounts, therefore that's the only behaviour that should be exposed to it through the type returned by salesAccount.Transactions.
The Accountant is programming against concrete types instead of interfaces. We can see that it depends on SalesAccount and ExpenseAccount. Again, violating encapsulation.
The solution - the Iterator pattern
A key principle in Object Oriented Programming (OOP) is to encapsulate what varies. The thing that's varying here is the way the Accountant is iterating over Transaction items due to the fact that different collections are being returned by the two account types. Therefore, following the stated principle, we need to encapsulate iteration.
Encapsulating iteration is the core idea behind the iterator pattern. To encapsulate a behaviour means to place the code that implements the behaviour into a type and hide the implementation details away from consumers of the type. So, what exactly is the behaviour behind iteration? To iterate through a collection means to get all of the items in the collection, one at a time. That's the essence of iteration and is what we need to encapsulate, and nothing more.
Encapsulating iteration
So, we need an object that encapsulates how we iterate through collections of objects. We're going to call this object an iterator.
To iterate over a collection, we basically need to be able to do two things:
- Check if the collection has any more items that haven't yet been seen.
- Get the next item from the collection.
We can capture that functionality in two methods: HasNext() and Next(), respectively.
Then we can go on to write code that looks like this
IIterator iterator = collection.CreateIterator(); // Ask a collection for an iterator
while(iterator.HasNext()) // While there are unseen items left
{
Item item = iterator.Next() // Get the next item from the collection
// Do something with the item here
}
A key aspect of the iterator pattern is that the collection itself provides an iterator object to clients that wish to iterate over the collection's items, therefore the returned iterator is specific to the collection that produced it. In other words, the returned iterator knows how to iterate over that collection type, the details of which are going to be different for different collections types, as we saw with a list versus an array.
Implementing the iterator pattern
We've established the methods required to perform iteration over a collection, so let's define an iterator interface with those as its members.
public interface IIterator<T>
{
bool HasNext();
T Next();
}
SalesAccount iterator implementation
Now, we need to define an iterator type that knows how to iterate over the Transaction items in a SalesAccount, this is going to implement our IIterator<T> interface, specifically IIterator<Transaction>.
public class SalesAccountIterator: IIterator<Transaction>
{
List<Transaction> transactions;
int position = 0;
public SalesAccountIterator(List<Transaction> transactions) => this.transactions = transactions;
public Transaction Next()
{
Transaction transaction = transactions[position];
position += 1;
return transaction;
}
public bool HasNext()
{
if (position >= transactions.Count)
{
return false;
}
else
{
return true;
}
}
}
The constructor takes a List<Transaction> - this is the list that this iterator is going to iterate over - and assigns the argument into a field. We have an int field called position, the purpose of this is to keep track of the index of the current item in the list, i.e. it's the mechanism the iterator will use to keep track of the items it has already seen. The Next() method simply gets the Transaction object at the current position, increments the position field, and returns the retrieved object; nothing surprising there. The HasNext() method checks if the iterator has seen all items in the list by checking if the position value is greater than the number of items in the list, if yes it returns false (i.e. "there are no more items left to see"), otherwise it returns true (i.e. "there are still items left to see").
Now we can work on integrating SalesAccountIterator into the SalesAccount. I said earlier that the Iterator pattern involves having the thing being iterated over provide an Iterator object to the thing that wishes to iterate over it. So, we need a method on our SalesAccount that does this, let's call it CreateIterator.
public class SalesAccount
{
private List<Transaction> transactions;
public SalesAccount()
{
transactions = new List<Transaction>();
AddTransaction(
name: "Stark Industries",
amount: 200000000,
taxRate: 0,
isReconciled: false
);
AddTransaction(
name: "Wayne Enterprises",
amount: 5500000,
taxRate: 10,
isReconciled: true
);
AddTransaction(
name: "Oscorp",
amount: 100000,
taxRate: 20,
isReconciled: false
);
}
public void AddTransaction(string name, float amount, float taxRate, bool isReconciled)
{
Transaction transaction = new Transaction(name, amount, taxRate, isReconciled);
this.transactions.Add(transaction);
}
public IIterator<Transaction> CreateIterator()
{
return new SalesAccountIterator(transactions);
}
}
The new SalesAccount class has a CreateIterator method with a return type of IIterator<Transaction>. Notice I haven't used a return type of SalesAccountIterator; we want to return the interface instead of the concrete class, since the client doesn't need to know how SalesAccountIterator is implemented, it only needs to know that an IIterator<Transaction> is returned in order to iterate through the items. Moreover, we - the people writing SalesAccount - don't want clients to know the implementation details. By providing an abstraction (IIterator<T>) we deliberately hide the implementation details. This means that if some point in the future we want to change how SalesAccount manages its data, we can completely change the iterator type without impacting any existing client code.
Notice also that the MenuItems property has been removed; we don't want a consumer of this class to see how SalesAccount is storing its items internally. SalesAccount is now storing its List<Transaction> in a private field. Reading the new definition of this class, you could interpret its intent as saying "if you want iterate over my Transactions, I'll give you an Iterator object that knows how - use that to do your iteration". So, to iterate over the items in the new SalesAccount, we use the iterator it provides by writing code like this:
IIterator<Transaction> iterator = SalesAccount.CreateIterator();
while(iterator.HasNext())
{
Transaction transaction = iterator.Next();
// Do something with transaction
}
ExpenseAccount iterator implementation
Next, let's do the same for the ExpenseAccount. First, we need to define an iterator that knows how to iterate over the Transactions in an ExpenseAccount.
public class ExpenseAccountIterator: IIterator<Transaction>
{
Transaction[] transactions;
int position = 0;
public ExpenseAccountIterator(Transaction[] transactions) => this.transactions = transactions;
public Transaction Next()
{
Transaction transaction = transactions[position];
position += 1;
return transaction;
}
public bool HasNext()
{
if (position >= transactions.Length || transactions[position] == null)
{
return false;
}
else
{
return true;
}
}
}
The code is very similar to the SalesAccountIterator class, except this one needs to know how to iterate over an array - Transaction[]. Since you need to set the size of an array up-front when you create it, its values are pre-set to null, therefore if you don't assign new objects into every element, some of them are going to remain as null. We therefore need to add a check for null in the HasNext() method when checking if the array has any unseen items left.
Now, we need to rework the ExpenseAccount class to provide an ExpenseAccountIterator to consumers that wish to iterate over its Transaction objects.
public class ExpenseAccount
{
private static int maxItems = 3;
private int numberOfItems = 0;
private Transaction[] transactions;
public ExpenseAccount()
{
transactions = new Transaction[maxItems];
AddTransaction(
name: "Gotham City Iron Works",
amount: -1500000,
taxRate: 10,
isReconciled: true
);
AddTransaction(
name: "Super Electronics",
amount: -100000,
taxRate: 20,
isReconciled: false
);
AddTransaction(
name: "Wakanda Vibranium Corporation",
amount: -100000000,
taxRate: 0,
isReconciled: true
);
}
public void AddTransaction(string name, float amount, float taxRate, bool isReconciled)
{
Transaction transaction = new (name, amount, taxRate, isReconciled);
if (numberOfItems >= maxItems)
{
Console.WriteLine("Account is full! Can't add transaction to account");
}
else
{
transactions[numberOfItems] = transaction;
numberOfItems += 1;
}
}
public IIterator<Transaction> CreateIterator()
{
return new ExpenseAccountIterator(transactions);
}
}
As with SalesAccountIterator, we have a new method, CreateIterator that returns an IIterator<Transaction>, and the internal collection is no longer being exposed through a public property, it's now being stored in private field.
Reworking Accountant to use new account types
The Accountant type now just needs to grab an IIterator<Transaction> from each of the account types and use it to iterate over the Transaction objects in each.
public class Accountant
{
private readonly ExpenseAccount expenseAccount;
private readonly SalesAccount salesAccount;
public Accountant (ExpenseAccount expenseAccount, SalesAccount salesAccount)
{
this.expenseAccount = expenseAccount;
this.salesAccount = salesAccount;
}
public void PrintTransactions()
{
IIterator<Transaction> salesIterator = salesAccount.CreateIterator();
IIterator<Transaction> expensesIterator = expenseAccount.CreateIterator();
Console.WriteLine("ACCOUNT\n----\nSALES");
PrintTransactions(salesIterator);
Console.WriteLine("\nEXPENSES");
PrintTransactions(expensesIterator);
}
public void PrintTransactions(IIterator<Transaction> iterator)
{
while(iterator.HasNext())
{
Transaction transaction = iterator.Next();
Console.WriteLine($"{transaction.Name}\n{transaction.Amount}\n{transaction.TaxRate}\n{transaction.IsReconciled}");
Console.WriteLine();
}
}
}
The new Accountant no longer requires a loop for each of the account objects, it has a single loop inside the overloaded PrintTransactions method that uses the iterator passed in to iterate over the Transaction objects and print the details of each. Plus, this code is now easily extendable: if we had another account type - LoanAccount, for example - we would just need to grab an iterator from it (IIterator<Transaction> loansIterator = loanAccount.CreateIterator();) in PrintTransactions and pass it to the overloaded PrintTransactions.
Notice Accountant no longer knows how account types are storing their Transactions internally. In the first PrintTransactions method it sees IIterator<Transaction> types, which is all it needs to perform iteration.
Further improvements
The changes made in this section aren't strictly part of the iterator pattern, but rather just fall under good OOP practice.
At the beginning of the post I mentioned that the initial code was bad because Accountant was programming to concrete types instead of interfaces; the changes made removed references to concrete collection types (List<Transaction> and Transaction[]), replacing them with the IIterator<Transaction> interface. However, the Accountant class is still referencing concrete account types in its constructor - ExpenseAccount and SalesAccount, so it'd be good to replace those with interfaces. Throughout the blog I've been referring to "account types", this indicates that we should have an interface representing account types, let's call it IAccount.
public interface IAccount
{
void AddTransaction(string name, float amount, float taxRate, bool isReconciled);
IIterator<Transaction> CreateIterator();
}
Now we can rework the account types to make them implement IAccount, then we can use that interface in place of the concrete types in Accountant.
public class SalesAccount: IAccount
{
private List<Transaction> transactions;
public SalesAccount()
{
transactions = new List<Transaction>();
AddTransaction(
name: "Stark Industries",
amount: 200000000,
taxRate: 0,
isReconciled: false
);
AddTransaction(
name: "Wayne Enterprises",
amount: 5500000,
taxRate: 10,
isReconciled: true
);
AddTransaction(
name: "Oscorp",
amount: 100000,
taxRate: 20,
isReconciled: false
);
}
public void AddTransaction(string name, float amount, float taxRate, bool isReconciled)
{
Transaction transaction = new Transaction(name, amount, taxRate, isReconciled);
this.transactions.Add(transaction);
}
public IIterator<Transaction> CreateIterator()
{
return new SalesAccountIterator(transactions);
}
}
public class ExpenseAccount: IAccount
{
private static int maxItems = 3;
private int numberOfItems = 0;
private Transaction[] transactions;
public ExpenseAccount()
{
transactions = new Transaction[maxItems];
AddTransaction(
name: "Gotham City Iron Works",
amount: -1500000,
taxRate: 10,
isReconciled: true
);
AddTransaction(
name: "Super Electronics",
amount: -100000,
taxRate: 20,
isReconciled: false
);
AddTransaction(
name: "Wakanda Vibranium Corporation",
amount: -100000000,
taxRate: 0,
isReconciled: true
);
}
public void AddTransaction(string name, float amount, float taxRate, bool isReconciled)
{
Transaction transaction = new (name, amount, taxRate, isReconciled);
if (numberOfItems >= maxItems)
{
Console.WriteLine("Account is full! Can't add transaction to account");
}
else
{
transactions[numberOfItems] = transaction;
numberOfItems += 1;
}
}
public IIterator<Transaction> CreateIterator()
{
return new ExpenseAccountIterator(transactions);
}
}
Now we can rework Accountant to work with the new account types.
public class Accountant
{
private IAccount expenseAccount;
private IAccount salesAccount;
public Accountant (IAccount expenseAccount, IAccount salesAccount)
{
this.expenseAccount = expenseAccount;
this.salesAccount = salesAccount;
}
public void PrintTransactions()
{
IIterator<Transaction> salesIterator = salesAccount.CreateIterator();
IIterator<Transaction> expensesIterator = expenseAccount.CreateIterator();
Console.WriteLine("ACCOUNT\n----\nSALES");
PrintTransactions(salesIterator);
Console.WriteLine("\nEXPENSES");
PrintTransactions(expensesIterator);
}
public void PrintTransactions(IIterator<Transaction> iterator)
{
while(iterator.HasNext())
{
Transaction transaction = iterator.Next();
Console.WriteLine($"{transaction.Name}\n{transaction.Amount}\n{transaction.TaxRate}\n{transaction.IsReconciled}");
Console.WriteLine();
}
}
}
Accountant now no longer has any references to concrete types.
In this post we covered the fundamentals of the iterator pattern, and implemented it from scratch. In the next post I'll show how C# has the iterator pattern built into the language and the .NET runtime libraries, which means you don't have to bother defining your own iterators. I'll also cover some other iteration-related language features like iterator methods and IAsyncEnumerable.
