Qbmove: the new email migration system developed by Qboxmail

Alessio Cecchi

Switching to a new email service provider entails meticulous planning for the migration of existing email accounts, along with all their associated folders and messages.

During the email migration planning process, it’s imperative to consider specific elements, such as:

Qboxmail’s Control Panel has consistently offered customers a user-friendly tool to strategically plan and automate IMAP-based email account migrations from their current provider to Qboxmail.

In October, a more sophisticated migration system, named Qbmove, was introduced. 

This system was meticulously developed by our dedicated Research and Development department. In contrast to its predecessor, which was relatively straightforward, clients would upload a CSV file, initiating a Perl process for each email account to handle message transfers via IMAP.

Nevertheless, the former migration system encountered several limitations:

As a response, we embarked on developing a new migration system that not only rectified these issues but also provided more adaptability for future implementations.

The architecture of the new migration service:

The idea behind the architecture of the new system is to have a single entry point, the orchestrator, which in turn configures the processes to be executed on other servers and tracks their progress.

The orchestrator is an application written in Nodejs, utilizing the SSH protocol to launch and monitor processes on the servers made available to it.

In this way, new servers can be added for executing migrations by simply specifying a new IP address to the orchestrator. It is also possible to dedicate entire server clusters to specific types of operations or clients using a highly flexible rules system.

Another fundamental detail is that once a process is initiated, it operates independently of the orchestrator. If the orchestrator were to malfunction or if we simply chose to update it, no ongoing migration process would be affected. This has always been a crucial requirement for us, especially since some migration processes can last for several days due to the slowness of the server from which we retrieve emails.

After the processes are launched, the Nodejs application monitors them, extracting useful data such as resource utilization, progress details, and any errors. These pieces of information are then exposed through internal APIs in our control panel, which are displayed to the user.

The servers responsible for the actual transfer do not require many dependencies but simply:

Migrations can be initiated on one server and completed on another, thanks to the sharing of states and data cache. Furthermore, we can allocate a higher number of parallel migrations for each client, thereby speeding up the migration process.

With Qbmove, we have the capability to introduce new features to provide greater flexibility in managing the email migration process for our customers.

We use cookies to provide you a better browsing experience, by continuing you accept their use. For more information visit the Privacy policy page.