Migrate data from Magento 1.x to Magento 2.x

This article shows how you can migrate data from Magento 1.x site to Magento 2.x site.

We will use the Magento’s official Data Migration tool module.

– The data migration can be done via the CLI commands.
– The data like categories, products, orders, configuration settings, etc. can be migrated.
-The data migration depends upon the set of rules defined in XML files.

Install the data migration tool module

This module assumes that:

– you already have a Magento 2 site installed
– you have NOT added any data in that Magento 2 site.

So, let’s check the version of your Magento 2 site:

• Open terminal/command-prompt
• Go to your Magento 2 site’s installation/root directory
• Run the following command:

php bin/magento --version

The output can be something like this:

Magento CLI version 2.2.5

This shows that I have 2.2.5 version of Magento.

Install data-migration-tool module from GitHub

• Open terminal
• Go to your Magento 2 site’s root directory
• Run the following command:

composer config repositories.data-migration-tool git https://github.com/magento/data-migration-tool
composer require magento/data-migration-tool:<version>

Here, <version> means the version of your Magento 2 site. In my case, as you can see above, I have version 2.2.5. Hence, the command for me will be:

composer require magento/data-migration-tool:2.2.5

This will ask for Username and Password for repo.magento.com.

To get the username and password for repo.magento.com, you have to generate access keys from https://marketplace.magento.com/customer/account/

– Go to https://marketplace.magento.com/customer/account/
– Login
– After you are logged in, click My Access Keys link
– On My Access Keys page, click on Generate Keys button
– Then, you will see two keys (Public and Private)
– Public Key will be your Username
– Private Key will be your Password

Install data-migration-tool module from repo.magento.com

• Open terminal
• Go to your Magento 2 site’s root directory
• Run the following command:

composer config repositories.magento composer https://repo.magento.com
composer require magento/data-migration-tool:<version>

Here, <version> means the version of your Magento 2 site. In my case, as you can see above, I have version 2.2.5. Hence, the command for me will be:

composer require magento/data-migration-tool:2.2.5

This will ask for Username and Password for repo.magento.com.

You can get the Username and Password by logging in to Magento Marketplace website and going to the Access Keys page. The steps are mentioned above.

Both installation processes (from github and from repo.magento.com) are quite similar. The only difference is the composer config command.

Enable the module

• Open terminal
• Go to your Magento 2 site’s root directory
• Run the following commands to enable the data-migration-tool module:

php bin/magento module:enable Magento_DataMigrationTool
php bin/magento setup:upgrade
php bin/magento clean:cache

Configure Magento Data Migration Tool

Mapping and Configuration files directory:

– Go to <Your-Magento2-root-directory>/vendor/magento/data-migration-tool/etc/

– You will see the following directories over there:
    – commerce-to-commerce (to migrate from Magento 1.x Enterprice Edition (EE) to Magento 2.x Enterprise Edition (EE))
    – opensource-to-commerce (to migrate from Magento 1.x Community Edition (CE) to Magento 2.x Enterprise Edition (EE))
    – opensource-to-opensource (to migrate from Magento 1.x Community Edition (CE) to Magento 2.x Community Edition (CE))

– There are some XSD files are well. They are the XML Schema files.
– For example, the config.xsd file is the Schema for the config.xml file present inside the above-mentioned folders.

mukesh:etc chapagain$ pwd
/Users/mukeshchapagain/Sites/magento2/vendor/magento/data-migration-tool/etc

mukesh:etc chapagain$ ls
class-map.xsd           groups.xsd          module.xml          settings.xsd
commerce-to-commerce        magento_path.php        opensource-to-commerce
config.xsd          map.xsd             opensource-to-opensource

Let us suppose that:

– you are migrating from Magento 1.x Community Edition (CE) to Magento 2.x Community Edition (CE)

– you are migrating from Magento version 1.9.3.8 to Magento version 2.x

To migrate from Magento 1.x CE to Magento 2.x CE, we have to go inside the folder opensource-to-opensource.

mukesh:etc chapagain$ pwd
/Users/mukeshchapagain/Sites/magento2/vendor/magento/data-migration-tool/etc

mukesh:etc chapagain$ cd opensource-to-opensource/

mukesh:opensource-to-opensource chapagain$ pwd
/Users/mukeshchapagain/Sites/magento2/vendor/magento/data-migration-tool/etc/opensource-to-opensource

mukesh:opensource-to-opensource chapagain$ ls
1.6.0.0                 1.9.2.2                 customer-document-groups.xml.dist
1.6.1.0                 1.9.2.3                 deltalog.xml.dist
1.6.2.0                 1.9.2.4                 eav-attribute-groups.xml.dist
1.7.0.0                 1.9.3.0                 eav-document-groups.xml.dist
1.7.0.1                 1.9.3.1                 log-document-groups.xml.dist
1.7.0.2                 1.9.3.2                 map-customer.xml.dist
1.8.0.0                 1.9.3.3                 map-document-groups.xml.dist
1.8.1.0                 1.9.3.4                 map-eav.xml.dist
1.9.0.0                 1.9.3.6                 map-log.xml.dist
1.9.0.1                 1.9.3.7                 map-stores.xml.dist
1.9.1.0                 1.9.3.8                 map-tier-price.xml.dist
1.9.1.1                 1.9.3.9                 order-grids-document-groups.xml.dist
1.9.2.0                 class-map.xml.dist          settings.xml.dist
1.9.2.1                 customer-attribute-groups.xml.dist

In the opensource-to-opensource folder, you will see folders for different Magento 1 version.

You will also see .dist files.

You can:

– either copy each of the .dist files and remove the .dist part from the name of the copied file
– or, you can directly rename .dist file and remove the .dist form its name

I will copy the .dist files and rename them. So, after copy & rename, we will have the following files inside the opensource-to-opensource folder:

mukesh:opensource-to-opensource chapagain$ ls
1.6.0.0                 1.9.3.2                 log-document-groups.xml
1.6.1.0                 1.9.3.3                 log-document-groups.xml.dist
1.6.2.0                 1.9.3.4                 map-customer.xml
1.7.0.0                 1.9.3.6                 map-customer.xml.dist
1.7.0.1                 1.9.3.7                 map-document-groups.xml
1.7.0.2                 1.9.3.8                 map-document-groups.xml.dist
1.8.0.0                 1.9.3.9                 map-eav.xml
1.8.1.0                 class-map.xml.dist          map-eav.xml.dist
1.9.0.0                 class-map.xml.xml           map-log.xml
1.9.0.1                 customer-attribute-groups.xml       map-log.xml.dist
1.9.1.0                 customer-attribute-groups.xml.dist  map-stores.xml
1.9.1.1                 customer-document-groups.xml        map-stores.xml.dist
1.9.2.0                 customer-document-groups.xml.dist   map-tier-price.xml
1.9.2.1                 deltalog.xml                map-tier-price.xml.dist
1.9.2.2                 deltalog.xml.dist           order-grids-document-groups.xml
1.9.2.3                 eav-attribute-groups.xml        order-grids-document-groups.xml.dist
1.9.2.4                 eav-attribute-groups.xml.dist       settings.xml
1.9.3.0                 eav-document-groups.xml         settings.xml.dist
1.9.3.1                 eav-document-groups.xml.dist

In this example, we are trying to Migrate from Magento 1.9.3.8 to Magento 2.x. Therefore, we will go inside the folder 1.9.3.8.

mukesh:opensource-to-opensource chapagain$ cd 1.9.3.8/

mukesh:1.9.3.8 chapagain$ pwd
/Users/mukeshchapagain/Sites/magento2/vendor/magento/data-migration-tool/etc/opensource-to-opensource/1.9.3.8

mukesh:1.9.3.8 chapagain$ ls
config.xml.dist     map-tier-price.xml.dist map.xml.dist

Inside the 1.9.3.8 folder, you can see three .dist files. Create a copy of them and rename them removing the .dist part from the name. After copy and rename, we will have the following files:

mukesh:1.9.3.8 chapagain$ ls
config.xml      config.xml.dist     map-tier-price.xml  map-tier-price.xml.dist map.xml         map.xml.dist

Edit config.xml file

From the folder 1.9.3.8, open config.xml file.

Edit Source & Destination Database Credentials

In the config.xml file, find the following code and add your source and destination database host, database name, database username, database password, and database port number.

source = your magento 1 site
destination = your magento 2 site

<source>
    <database host="localhost" name="magento1-db-name" user="magento1-db-username" password="magento1-db-password" />
</source>
<destination>
    <database host="localhost" name="magento2-db-name" user="magento2-db-username" password="magento2-db-password" />
</destination>

Rename mapping files name

In the same config.xml file, under “options” node, there’s the list of mapping file names. The file names have .dist in them. You just have to remove the .dist text from the files name.

BEFORE

<options>
    <map_file>etc/opensource-to-opensource/1.9.3.8/map.xml.dist</map_file>
    <eav_map_file>etc/opensource-to-opensource/map-eav.xml.dist</eav_map_file>
    ...
    ...

AFTER RENAME

<options>
    <map_file>etc/opensource-to-opensource/1.9.3.8/map.xml</map_file>
    <eav_map_file>etc/opensource-to-opensource/map-eav.xml</eav_map_file>
    <eav_document_groups_file>etc/opensource-to-opensource/eav-document-groups.xml</eav_document_groups_file>
    <eav_attribute_groups_file>etc/opensource-to-opensource/eav-attribute-groups.xml</eav_attribute_groups_file>
    <log_map_file>etc/opensource-to-opensource/map-log.xml</log_map_file>
    <log_document_groups_file>etc/opensource-to-opensource/log-document-groups.xml</log_document_groups_file>
    <settings_map_file>etc/opensource-to-opensource/settings.xml</settings_map_file>
    <customer_map_file>etc/opensource-to-opensource/map-customer.xml</customer_map_file>
    <customer_document_groups_file>etc/opensource-to-opensource/customer-document-groups.xml</customer_document_groups_file>
    <customer_attribute_groups_file>etc/opensource-to-opensource/customer-attribute-groups.xml</customer_attribute_groups_file>
    <delta_document_groups_file>etc/opensource-to-opensource/deltalog.xml</delta_document_groups_file>
    <order_grids_document_groups_file>etc/opensource-to-opensource/order-grids-document-groups.xml</order_grids_document_groups_file>
    <map_document_groups>etc/opensource-to-opensource/map-document-groups.xml</map_document_groups>
    <class_map>etc/opensource-to-opensource/class-map.xml</class_map>
    <tier_price_map_file>etc/opensource-to-opensource/1.9.3.8/map-tier-price.xml</tier_price_map_file>
    <stores_map_file>etc/opensource-to-opensource/map-stores.xml</stores_map_file>

Add database table prefix

If your source (Magento 1.x) or destination (Magento 2.x) database tables have prefix names then you need to specify them in the config.xml file. There’s XML nodes for source_prefix and dest_prefix.

If your database tables do not have prefix then you can simply ignore this step.

<source_prefix />
<dest_prefix />

Add crypt key

In the config.xml file, you need to add the encryption key of your Magento 1.x site.

The encryption key of your Magento 1.x site is found in the file: app/etc/local.xml

<crypt_key>abxcadfa232sdasdf</crypt_key>

Be prepared for migration

You need to be sure that no data is being inserted into the Magento 1.x database while you are migrating the data from Magento 1.x to Magento 2.x.

Hence,

• you need to stop all the cron jobs for Magento 1.x site
• put the Magento 1.x site in maintenance mode

Migrate data

The migration command format is as under:

php bin/magento migrate:<mode> [-r|--reset] {<path to config.xml>}

where:

<mode> may be: settings, data, or delta
[-r|--reset] is an optional argument that starts the migration from the beginning
{<path to config.xml>} is the absolute file system path to config.xml; this is a required argument

The sequence of migration is:

– First of all, we need to migrate settings.
– Then, we migrate data.
– After that, we migrate the changes if there are any.

Migrate Settings

First of all, we need to migrate different settings. We will run a command that will migrate configuration settings, store settings, website settings from Magento 1.x to Magento 2.x.

• Go to your Magento 2 root directory.

mukesh:magento2 chapagain$ pwd
/Users/mukeshchapagain/Sites/magento2

• Run the following command:

php bin/magento migrate:settings --reset vendor/magento/data-migration-tool/etc/opensource-to-opensource/1.9.3.8/config.xml

Migrate Data

We will now run the command to migrate different data like products, categories, orders, customers from Magento 1.x. to Magento 2.x.

• Go to your Magento 2 root directory.

mukesh:magento2 chapagain$ pwd
/Users/mukeshchapagain/Sites/magento2

• Run the following command:

php bin/magento migrate:data vendor/magento/data-migration-tool/etc/opensource-to-opensource/1.9.3.8/config.xml

While running this command, if you get the errors like:

Source documents are not mapped
Destination documents are not mapped
Source fields are not mapped
Destination fields are not mapped

Documents mean database tables in this case.

This means that the tables / fields present in source (Magento 1.x site) are not present in destination (Magento 2.x site).

To resolve this error, you need to add the ignore field or ignore document code in the map.xml file.

<source>
    <document_rules>
        <ignore>
            <document>table_name</document>
        </ignore>
        ...

<destination>
    <document_rules>
        <ignore>
            <document>table_name</document>
        </ignore>
        ...

<source>
    <field_rules>
        <ignore>
            <document>field_name</document>
        </ignore>
        ...

<destination>
    <field_rules>
        <ignore>
            <document>field_name</document>
        </ignore>
        ...

Migrate Changes / Incremental Migration

Incremental Migration means only migrating those data changes that have been done since we migrated the data last time.

After you have migrated the data from Magento 1.x to Magento 2.x and if there have been some changes in the data in your Magento 1.x site then you can use this command to migrate only those recent changes from your Magento 1.x site to Magento 2.x site.

• Go to your Magento 2 root directory.

mukesh:magento2 chapagain$ pwd
/Users/mukeshchapagain/Sites/magento2

• Run the following command:

php bin/magento migrate:delta vendor/magento/data-migration-tool/etc/opensource-to-opensource/1.9.3.8/config.xml

That is all about the Official Magento data migration tool module.

Here’s a link to troubleshoot some of the common error message that you might face during data migration: https://devdocs.magento.com/guides/v2.1/migration/migration-troubleshooting.html

Hope this helps. Thanks.