WHAT IS DATA?
Computer data is information processed or stored by a computer. This information may be in the form of text documents, images, audio clips, software programs, or other types of data. Computer data may be processed by the computer’s CPU and is stored in files and folders on the computer’s hard disk.
At its most rudimentary level, computer data is a bunch of ones and zeros, known as binary data. Because all computer data is in binary format, it can be created, processed, saved, and stored digitally. This allows data to be transferred from one computer to another using a network connection or various media devices. It also does not deteriorate over time or lose quality after being used multiple times.
Data are units of information, often numeric, that are collected through observation. In a more technical sense, data are a set of values of qualitative or quantitative variables about one or more persons or objects, while a datum is a single value of a single variable.
TYPES OF DATA
Qualitative Data Type
Qualitative or Categorical Data describes the object under consideration using a finite set of discrete classes. It means that this type of data can’t be counted or measured easily using numbers and therefore divided into categories. The gender of a person (male, female, or others) is a good example of this data type.
These are usually extracted from audio, images, or text medium. Another example can be of a smartphone brand that provides information about the current rating, the color of the phone, category of the phone, and so on. All this information can be categorized as Qualitative data. There are two subcategories under this:
Nominal
These are the set of values that don’t possess a natural ordering. Let’s understand this with some examples. The color of a smartphone can be considered as a nominal data type as we can’t compare one color with others.
It is not possible to state that ‘Red’ is greater than ‘Blue’. The gender of a person is another one where we can’t differentiate between male, female, or others. Mobile phone categories whether it is midrange, budget segment, or premium smartphone is also nominal data type.
Ordinal
These types of values have a natural ordering while maintaining their class of values. If we consider the size of a clothing brand then we can easily sort them according to their name tag in the order of small < medium < large. The grading system while marking candidates in a test can also be considered as an ordinal data type where A+ is definitely better than B grade.
These categories help us deciding which encoding strategy can be applied to which type of data. Data encoding for Qualitative data is important because machine learning models can’t handle these values directly and needed to be converted to numerical types as the models are mathematical in nature.
For nominal data type where there is no comparison among the categories, one-hot encoding can be applied which is similar to binary coding considering there are in less number and for the ordinal data type, label encoding can be applied which is a form of integer encoding.
WHAT IS DATA MIGRATION?
Data migration is the process of moving data from one location to another, one format to another, or one application to another. Generally, this is the result of introducing a new system or location for the data. The business driver is usually an application migration or consolidation in which legacy systems are replaced or augmented by new applications that will share the same dataset. These days, data migrations are often started as firms move from on-premises infrastructure and applications to cloud-based storage and applications to optimize or transform their company.
Types of data migration
There are numerous business advantages to upgrading systems or extending a data center into the cloud. For many firms, this is a very natural evolution. Companies using cloud are hoping that they can focus their staff on business priorities, fuel top-line growth, increase agility, reduce capital expenses, and pay for only what they need on demand. However, the type of migration undertaken will determine how much IT staff time can be freed to work on other projects.
First, let’s define the types of migration:
- Storage migration. The process of moving data off existing arrays into more modern ones that enable other systems to access it. Offers significantly faster performance and more cost-effective scaling while enabling expected data management features such as cloning, snapshots, and backup and disaster recovery.
- Cloud migration. The process of moving data, application, or other business elements from either an on-premises data center to a cloud or from one cloud to another. In many cases, it also entails a storage migration.
- Application migration. The process of moving an application program from one environment to another. May include moving the entire application from an on-premises IT center to a cloud, moving between clouds, or simply moving the application’s underlying data to a new form of the application hosted by a software provider.
How to plan a data migration
Data migration involves 3 basic steps:
- Extract data
- Transform data
- Load data
Moving important or sensitive data and decommissioning legacy systems can put stakeholders on edge. Having a solid plan is a must; however, you don’t have to reinvent the wheel. You can find numerous sample data migration plans and checklists on the web. For example, Data migration pro, a community of data migration specialists, has a comprehensive checklist that outlines a 7-phase process:
- Premigration planning. Evaluate the data being moved for stability.
- Project initiation. Identify and brief key stakeholders.
- Landscape analysis. Establish a robust data quality rules management process and brief the business on the goals of the project, including shutting down legacy systems.
- Solution design. Determine what data to move, and the quality of that data before and after the move.
- Build & test. Code the migration logic and test the migration with a mirror of the production environment.
- Execute & validate. Demonstrate that the migration has complied with requirements and that the data moved is viable for business use.
- Decommission & monitor. Shut down and dispose of old systems.
This may appear to be an overwhelming amount of work, but not all these steps are needed for every migration. Each situation is unique, and each company approaches the task differently.
DATA MIGRATION TOOLS
There are three primary types of data migration tools to consider when migrating your data:
- On-premise tools. Designed to migrate data within the network of a large or medium Enterprise installation.
- Open Source tools. Community-supported and developed data migration tools that can be free or very low cost.
- Cloud-based tools. Designed to move data to the cloud from various sources and streams, including on-premise and cloud-based data stores, applications, services, etc.
On-premise data migration tools
On-premise solutions are designed to migrate data between two or more servers or databases within a large or medium enterprise/network, without moving data to the cloud. These solutions are optimal if you are performing tasks like changing data warehouses or moving the location of your primary data store, or if you are simply bringing together data from disparate sources on-premise. Some companies prefer on-premise solutions due to security restrictions.
Open source data migration tools
Open source is software you can use, modify, and share because its design is publicly accessible. Typically, open source solutions are free or lower in cost than commercial alternatives. Sometimes commercial products are built on open source products and/or offer an open source, limited version for download. Open source data migration tools can be a practical option for migrating your data, especially if your project is not large or complex. However, to work with open source, you may need some coding skills.
The following list shows some popular open source data migration tools:
Cloud-based data migration tools
Cloud-based data migration solutions are the latest generation and are designed to move data to the cloud, either from an on-premise store, an application or stream, or another cloud-based store. Cloud-based solutions are optimal if you are already storing your data in the cloud or if you intend to move your data to the cloud. Many companies see cost efficiencies and increased security in moving data from on-premise to the cloud and need a data migration tool to help with this process. And, cloud-based data migration tools tend to be very flexible about the types of data they can handle.
Who is a data migration analyst?
Perform source system data analysis in order to manage source to target data mapping. … Perform migration and testing of static data and transaction data from one core system to another. Perform data migration audit, reconciliation and exception reporting.
Data Migration Analyst Responsibilities and Duties.
THE FOLLOWING ARE THE DUTIES OF A DATA MIGRATION ANALYST
Work with customers in gathering business requirements for data migration needs.
Work across multiple functional projects to understand data usage and implications for data migration.
Assist in designing, planning and managing the data migration process.
Work with subject matter experts and project team to identify, define, collate, document and communicate the data migration requirements.
Prepare data migration plans including migration risk, milestones, quality and business sign-off details.
Manage assigned risks and monitor potential impacts as part of the data migration plan.
Develop best practice, processes, and standards for effectively carrying out data migration activities.
Perform source system data analysis in order to manage source to target data mapping.
Perform migration and testing of static data and transaction data from one core system to another.
Perform data migration audit, reconciliation and exception reporting.
Manage cross-program data assurance for physical data items in source and target systems.
Drop your comment
