Methods to automate magmi

Question

Can anyone tell me how to automate magmi to perform import on scheduled time every day.

I have heard that it can be done via cli but dont know how to use cli.

Please give me a step wise procedure about how to cli and what commands to use for automating imports.

I saw magmi wiki site could not understand much about how to use cli.

Please give me a proper solution about how can i automate magmi.

I even tried to use below link but it is now working

wget "http://user:password@example.com/magmi/web/magmi_run.php?mode=create&profile=default&engine=magmi_productimportengine:Magmi_ProductImportEngine&CSV:filename=/magmitest.csv" -O /dev/null

sickelap · Answer 1 · 2013-05-05T18:03:43.583

If you can go with system cron (cli version for your problem) then here's complete solution that I'm using in one of my project (simplified version).

I will be using Company as vendor name and module name will be Magmi.

First step is to install magmi as usual. And I think you already have it installed.

Next, create app/etc/modules/Company_Magmi.xml with following content

<?xml version="1.0"?>
<config>
    <modules>
        <Company_Magmi>
            <active>true</active>
            <codePool>local</codePool>
            <version>0.0.1</version>
        </Company_Magmi>
    </modules>
</config>

Then create app/code/local/Company/Magmi/etc/config.xml with the following content

<?xml version="1.0"?>
<config>
    <modules>
        <Company_Magmi>
            <version>0.0.1</version>
        </Company_Magmi>
    </modules>
    <global>
        <models>
            <company_magmi>
                <class>Company_Magmi</class>
            </company_magmi>
        </models>
    </global>
    <crontab>
        <jobs>
            <magmi_update>
                <schedule>
                    <cron_expr>*/5 * * * *</cron_expr>
                </schedule>
                <run>
                    <model>company_magmi/cron::magmiUpdate</model>
                </run>
            </magmi_update>
        </jobs>
    </crontab>
</config>

Create app/code/local/Company/Magmi/Cron.php file with the following content

<?php

require_once(dirname(__FILE__) . "/../../../../../magmi/plugins/inc/magmi_datasource.php");
require_once(dirname(__FILE__) . "/../../../../../magmi/integration/productimport_datapump.php");

class Company_Magmi_Cron {

    public function magmiUpdate()
    {
        $items = array(); // build your own list of items to create/update

        $this->import($items);
    }

    private function import($items, $mode = 'create', $indexes = 'all')
    {
        if (count($items) > 0) {
            $dp = new Magmi_ProductImport_DataPump();
            $dp->beginImportSession("PROFILE_NAME", $mode);
            foreach ($items as $item) {
                $dp->ingest($item);
            }
            $dp->endImportSession();
            $this->reindex($indexes);
        }
    }

    private function reindex($string = 'all')
    {
        /** @var $indexer Mage_Index_Model_Indexer */
        $indexer = Mage::getModel('index/indexer');

        $processes = array();

        if ($string == 'all') {
            $processes = $indexer->getProcessesCollection();
        } else {
            $codes = explode(',', $string);
            foreach ($codes as $code) {
                $process = $indexer->getProcessByCode(trim($code));
                if ($process) {
                    $processes[] = $process;
                }
            }
        }

        /** @var $process Mage_Index_Model_Process */
        foreach ($processes as $process) {
            $process->reindexEverything();
        }
    }
}

And finally change PROFILE_NAME to your profile name in magmi.

Having all that in place you will have to build the list of items to create/update. It's really simple. Here's an example:

Say, you want to update stock for the products. You would create CSV file like this:

sku,qty
"SOMESKU","10"
"SNOTHERSKU","2"

Just build $items like this:

$items[] = array(
    "sku" => "SOMESKU",
    "qty" => "10"
);
$items[] = array(
    "sku" => "ANOTHERSKU",
    "qty" => "2"
);

And don't forget to setup cron for Magento!

You've got the idea, right?

That's it.

But do i have to built this array specifically every time I perform import. I have created this for my client. He has no programming idea. — Satyendra Mishra, May 06 '13 at 07:12
Yes, but you will do that anyway when creating CSV file. Instead of that you should transform your CSV building logic into creating $items array. In my case I'm fetching data from external WebService. — sickelap, May 06 '13 at 07:34
We've set this up following the above instructions, but are receiving this message below with a cron error sorry. Any ideas on how to debug/rectify? Cron error while executing magmi_update: exception 'Mage_Core_Exception' with message 'Invalid callback: company_magmi/cron::magmi_update does not exist' in /home/sites/galaxystores.co.uk/public_html/app/Mage.php:595 — MagentoMac, Feb 24 '15 at 23:55

score 2 · Accepted Answer · answered Jun 06 '13 at 13:42

2

we can also automate via curl command its quite easy by that

$url="http://learning.iksuladev.com/satyendra/magmi/web/magmi_run.php?mode=create&profile=satyendra&engine=magmi_productimportengine:Magmi_ProductImportEngine&CSV:filename=../../var/import/sample.csv -O /dev/null";

$ch = curl_init($Url);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, '');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$result = curl_exec($ch);
curl_close($ch);

answered Jun 06 '13 at 13:42

Satyendra Mishra

523
2
9
21

i have done same just change profile name , mode and file name but not working. can you please tell me what should i change which is remaining ? – Sarfaraj Sipai Jan 09 '18 at 11:05

score 0 · Answer 3 · edited Jul 23 '15 at 12:36

0

Adding to the above answer: if above required files not work please use below files.

require_once(dirname(__FILE__) . "/../../../../../magmi/inc/magmi_defs.php");
require_once(dirname(__FILE__) . "/../../../../../magmi/integration/inc/productimport_datapump.php");

it worked for me with these files.

edited Jul 23 '15 at 12:36

Stuart Siegler

1,686
4
30
38

answered Jul 23 '15 at 11:14

Ranjith

1

Once you have more reputation, this would likely be added as a comment to the post or answer, above. Until then, welcome to Stackoverflow.... – Stuart Siegler Jul 23 '15 at 11:42

Oli Girling · Answer 4 · 2018-07-09T15:50:32.177

Using @sickelap's code as a base and @Ranjith comment regarding the required files, ive just created an extension for Magento that allows you to run Magmi via Magento's cron. Hope this helps

Download it free here -> https://github.com/cameraki/Magento-Magmi-Via-Cron-Extension/

As comments mentioned, makes sense I post the code here in case the link goes offline, so here it is. ps, since I made a readme with the extension on github I have added this information at the bottom to help you get started. Again thanks to @sickelap and @Ranjith as this was built on their answers

app/code/local/Oli/Magmi/Cron.php

<?php
/**
 * Created by PhpStorm.
 * User: Oli
 * Date: 02/11/2017
 * Time: 17:05
 */

/* Change importer to the name of your Magmi directory */
require_once(Mage::getBaseDir() . "/importer/plugins/inc/magmi_datasource.php");
require_once(Mage::getBaseDir() . "/importer/integration/inc/productimport_datapump.php");

class Oli_Magmi_Cron {

    const LOG_DIRECTORY = "oli-extensions.log";

    public function runMagmi()
    {


        try {
            $directory = Mage::getBaseDir('var') . '/import/';
            $magmiCsvFile = $directory . 'magmiUploadReady.csv';

            if(!file_exists($magmiCsvFile)) {
                throw new Exception('CSV file for Magmi to upload cant be found.');
            }
        } catch (Exception $e) {
            $message = date('F jS (l) Y h:i:s A') . " === MAGMI (" . __METHOD__ . ") === Could not load todays magmiupload file. It must not have run today. This script will now stop: ".$e->getMessage();
            Mage::log($message, null, self::LOG_DIRECTORY);
            die();
        }

        $productArray = $this->csvToArray($magmiCsvFile);

        $indexes = 'catalog_product_attribute,catalog_product_price,cataloginventory_stock';
        $mode = 'update';
        $profile = 'cronUpdatePrices';
        try {
            $this->import($productArray, $mode, $profile);
        } catch (Exception $e) {
            $message = date('F jS (l) Y h:i:s A') . " === MAGMI (" . __METHOD__ . ") === There was an issue importing the products: ".$e->getMessage();
            Mage::log($message, null, self::LOG_DIRECTORY);
        }
        try {
            $this->reindex($indexes);
        } catch (Exception $e) {
            $message = date('F jS (l) Y h:i:s A') . " === MAGMI (" . __METHOD__ . ") === There is a problem with the reindex function: ".$e->getMessage();
            Mage::log($message, null, self::LOG_DIRECTORY);
        }
        /* log that it actually ran */
        $message = date('F jS (l) Y h:i:s A') . " === MAGMI (" . __METHOD__ . ") === Has finished running";
        Mage::log($message, null, self::LOG_DIRECTORY);

    }



    public function csvToArray($filename)
    {
        $productArray = array_map('str_getcsv', file($filename));
        array_shift($productArray);
        $newProductArray = array();
        /* Customise the item array index's to match the file you are importing */
        foreach ($productArray as $key => $item) {
            $newProductArray[$key]['sku'] = $item[0];
            $newProductArray[$key]['buy_price'] = $item[1];
            $newProductArray[$key]['price'] = $item[2];
            $newProductArray[$key]['store'] = $item[3];
            $newProductArray[$key]['price_updated'] = $item[4];
            $newProductArray[$key]['amazon_euros'] = $item[5];
            $newProductArray[$key]['amazon_dollars'] = $item[6];
            $newProductArray[$key]['qty'] = $item[7];
            $newProductArray[$key]['is_in_stock'] = $item[8];

        }
        return $newProductArray;
    }



    private function import($items, $mode, $profile)
    {
        if (count($items) > 0) {
            try{

                $dp = new Magmi_ProductImport_DataPump();
                $dp->beginImportSession($profile, $mode);
                foreach ($items as $item) {
                    $dp->ingest($item);
                }
                $dp->endImportSession();

            } catch (Exception $e) {
                $message = date('F jS (l) Y h:i:s A') . " === MAGMI (" . __METHOD__ . ") === Seems to be an issue with the import function: ".$e->getMessage();
                Mage::log($message, null, self::LOG_DIRECTORY);
            }
        }
    }

    private function reindex($string)
    {
        /** @var $indexer Mage_Index_Model_Indexer */
        $indexer = Mage::getModel('index/indexer');

        $processes = array();

        if ($string == 'all') {
            $processes = $indexer->getProcessesCollection();
        } else {
            $codes = explode(',', $string);
            foreach ($codes as $code) {
                $process = $indexer->getProcessByCode(trim($code));
                if ($process) {
                    $processes[] = $process;
                }
            }
        }

        /** @var $process Mage_Index_Model_Process */
        foreach ($processes as $process) {
            $process->reindexEverything();
        }
    }
}

app/code/local/Oli/Magmi/etc/config.xml

<?xml version="1.0"?>
<config>
    <modules>
        <Oli_Magmi>
            <version>0.0.1</version>
        </Oli_Magmi>
    </modules>
    <global>
        <models>
            <oli_magmi>
                <class>Oli_Magmi</class>
            </oli_magmi>
        </models>
    </global>
    <crontab>
        <jobs>
            <update_prices_via_magmi>
                <schedule>
                    <cron_expr>30 3 * * 2</cron_expr>
                </schedule>
                <run>
                    <model>oli_magmi/cron::runMagmi</model>
                </run>
            </update_prices_via_magmi>
        </jobs>
    </crontab>
</config>

app/etc/modules/Oli_Magmi.xml

<?xml version="1.0"?>
<config>
    <modules>
        <Oli_Magmi>
            <active>true</active>
            <codePool>local</codePool>
        </Oli_Magmi>
    </modules>
</config>

Magento Magmi Via Cron Extension

Magento extension to run Magmi via Magento's cron from a CSV file

Note: This only works if you have Magmi (Magento Mass Importer)

What does this do?

Magmi allows you to make changes to products in bulk and fast. Its fast because its a direct import into the database. So use Magmi with caution and backup your database before you import.

This extension allows you to run Magmi automatically using Magento's cron. Great for updating product pricing automatically every day, or changing stock levels in bulk once a week.

Can I use it out the box?

You need to make sure you have Magmi on your server. Then:

On line 10 & 11 of app/code/local/Oli/Magmi/Cron.php change the "/importer/...." to the location of your Magmi install. Eg, "/magmi/...."
On line 23 of app/code/local/Oli/Magmi/Cron.php change $magmiCsvFil to be the address of the CVS file you are importing from. Once you have changed this, change the lines around line 66 to map the columns from the CSV file into an array. The keys of $newProductArray[$key]['...'] should match the attributes you are changing. Eg, if you want to update stock levels, you will use $newProductArray[$key]['qty'] since 'qty' is what Magento's stock levels are.

Customisation

On line 36 of app/code/local/Oli/Magmi/Cron.php this sets which Magento settings you want to re-index. The ones I add are: catalog_product_attribute catalog_product_price cataloginventory_stock You may want to add more or change these.
On line 37 of app/code/local/Oli/Magmi/Cron.php this is Magmi mode. "create" creates and updates items, "update" updates only, "xcreate" creates only.
On line 38 of app/code/local/Oli/Magmi/Cron.php change this to match the name of the profile you have created in Magmi. You can use 'default' if you want to use the default profile however this is worth looking at because you want to make sure On the Fly indexing is turned off becauase this extension re-index's your magento store at the end. If you use a profile that has On the Fly indexing, it will re-index after each product, taking up a lot of server load for no reason.

More info You can find more information about the Magmi DataDump API here -> http://wiki.magmi.org/index.php/Magmi_Datapump_API

Please don't just post some tool or library as an answer. At least demonstrate [how it solves the problem](http://meta.stackoverflow.com/a/251605) in the answer itself. — Zoe, Jul 09 '18 at 13:52
I have updated the answer to provide the code. Hope this is enough to remove the downvote? — Oli Girling, Jul 10 '18 at 09:09

Methods to automate magmi

4 Answers4