Publications / Why it is worth transferring big data to cloud services and how to do it painlessly

The migration of the university's critical digital infrastructure to the cloud is an asterisk task. But an example of the largest technical university in Ukraine is the Igor Sikorsky Kyiv Polytechnic Institute. Igor Sikorsky - proves that even in the conditions of war and blackouts, a similar task can be completed quickly and with the help of a small team. It is only necessary to avoid following the beaten bureaucratic paths but to find the community's support and set priorities correctly. How KPI moved to the cloud, says Ph.D. IT consultant Andrey Gubsky, who helped the university transfer data.

There is a joke among IT people that people are divided into those who make backups and those who already make backups. Many have moved to the "already" camp because of the winter attacks on Ukrainian energy infrastructure.

At the same time, some of the data was secured earlier. Last summer, Amazon announced that during the first months of the full-scale invasion of the Russian Federation into Ukraine, more than 10 PB of data from government agencies, banking, and educational systems of Ukraine migrated to its services.

Big Data is a Big Challenge

The autumn of 2022 raised the issue of data storage Igor Sikorsky Kyiv Polytechnic Institute in cloud services. The first talk about such a need began in 2011. But this became a prominent and urgent issue after the shelling and power outages.

Due to power outages, various university services began to fail. It was necessary to guarantee access to at least critical ones related to the educational process.

Transferring the data of one of the largest universities in the country is a big challenge. However, by the end of 2022, we established cooperation with Amazon and prepared and transferred KPI data to Amazon Web Services. And to do it in such a way as to be able, if necessary, to go through a similar migration again, but in a shorter time. But in the process, we ran into several obstacles.

Challenges and Solutions

What are these challenges, and how can they be overcome? It will help save time, nerve cells, and money for those who also decided to transfer their data.

Use the Power of Weak Ties

When the question arose about the uninterrupted operation of the infrastructure during outages, there were two options: install generators on their capacities or transfer data to the cloud.

However, university mechanisms could be faster and more relaxed. Request after request - and any of these options, if initiated by the university, could have dragged on for a long time, and the problem was urgent. Therefore, we decided to seek help directly from the Ministry of Digital Development, which already then had successful experience in such processes.

On the recommendation of my close friends, we got in touch with Anton Melnik, Advisor to the Minister of Digital Transformation, who is responsible for communication with technology corporations. He quickly responded and contacted us with Amazon. We discussed organizing the transfer with the Amazon PL team a week later.

Conclusion - in critical situations, it is sometimes important to act not following the organization's hierarchical structure but following the path of direct communications. The strength of such contacts is to save time and nerves.

If you need help, talk about it, in person or on social media. If you are also a non-profit organization, you have certain advantages by default: government agencies, consultants, or potential partners are usually even more open to cooperation with public and non-governmental initiatives.

Move Data Smartly

After agreeing on the technical details, Amazon Web Services provided KPI with a grant for UAH 500,000 in the form of virtual loans to a special university account. The grant covered the six months allotted for the implementation of the project.

It was about the rapid transfer of critical components to the cloud. Therefore, we initially set priorities and decided to transfer data on which the current educational process and the university's work directly depended.

But an apparent key problem arose - technical debt, particularly the incompatibility of outdated university software with cloud service components. There needed to be more time and engineering resources to develop an additional software product.

Therefore, we developed a sequential migration plan and applied Docker containerization technology. This industry standard is supported by all major cloud providers such as Azure, AWS, GCP, or DigitalOcean.

The process of deploying a container is quite simple and is suitable for dealing with the technical debt of legacy software. And the main bonus is the speed of deployment of such software.

If the data has to be transferred to another cloud service, this process for the same volume will already take not months but days or a maximum of a week.

Our team of four people prepared critical KPI services for work in the cloud environment in a month and transferred them to the cloud. The following requirements were the basis for this:

• Make only the minimum number of essential changes to existing services.
• Fully use the capabilities of the cloud platform.
• Make the final technological solution understandable for the university staff accompanying it.

So a month later, the KPI data was already on AWS.

Essential Tips and Tricks for an Easy Transition to the Cloud

Among the main takeaways from this project is that before you even begin the data migration process, prioritize. Determine how much data is critical to transfer - it can be some things.

It is better to choose cloud solutions that are more common on the market: they are easier to use at different sites, and it is easier to find the necessary engineers.

Consider cloud storage through the lens of the next steps. The data and the team must be ready for the next possible "move" and be able to do it quickly.

If you make up key tips for business owners and, for example, university leaders who are preparing to migrate data to the cloud, the list would be as follows:

1. List critical and non-critical services for your business processes.
2. Assess the complexity of migrating to the cloud for each service.
3. Migrate sequentially:
• critical services that are easy to migrate;
• essential services that are difficult to migrate;
• non-critical services that are easy to migrate;
• non-critical services that are difficult to migrate.
4. Try to clear only some of your technical debt at a time so you can roll over everything at a time. Do it evenly.

The memory of recent blackouts fades quickly, which is annoying. It seems to many that nothing threatens the data so much. But it's not just about preparing for a power outage. Some data security threats in Ukraine are independent of the season. These are Russian cybercriminals or the possibility of physically destroying servers due to rocket attacks.

Therefore, for entrepreneurs, heads of government agencies, and universities who still store data or their backups only on servers in Ukraine, now is the perfect time to think about the cloud. Implementing is easier because there is no time pressure and additional stress due to blackouts. Many foundations and businesses are helping other organizations with funding or the data migration process.

The study already implemented successful cases to avoid mistakes and save time. Feel free to google, contact and venture into such projects.