0

Hi I'm trying to make a cron job for importing products into my magento. I've got it all working just on an ad hoc basis but now I need a cron.

Ideally I would like to use the wget as that's the most straightforward.

So I'm using wget "http://www.xxxxxx.com/magmi/web/magmi_run.php?profile=default&mode=xcreate&engine=magmi_productimportengine:Magmi_ProductImportEngine" -O /dev/null

but I'm having problems with this error message HTTP request sent, awaiting response... Read error (Connection reset by peer) in headers. Coming up - which trys for 20 times and then gives up.

Can anyone tell me what the problem could be? Thanks Richard

Richard Housham
  • 864
  • 2
  • 15
  • 34

3 Answers3

4

I would recommend using the Magmi CLI Interface to handle cron job running (as HTTP connections may have timeouts and such).

I would run the following command on your cron job:

php /path/to/magmi/cli/magmi.cli.php -mode=create

You can define a custom profile and mode by referencing the documentation here: http://sourceforge.net/apps/mediawiki/magmi/index.php?title=Magmi_command_line

dweeves
  • 5,525
  • 22
  • 28
Axel
  • 10,732
  • 2
  • 30
  • 43
  • Thanks I just tried that but got this error PHP Notice: Undefined offset: -1 in /var/www/vhosts/deal-buster.co.uk/httpdocs/magmi/inc/magmi_engine.php on line 211 startup:Performing Datasouce Lookup... PHP Fatal error: Call to a member function getRecordsCount() on a non-object in /var/www/vhosts/deal-buster.co.uk/httpdocs/magmi/engines/magmi_productimportengine.php on line 1309 – Richard Housham Nov 29 '12 at 09:44
  • I think this was me and the wrong url - seems to be ok now - thanks for listening to an old hand! – Richard Housham Nov 30 '12 at 09:31
2

Ive just created an extension for Magento that allows you to run Magmi via Magento's cron. This maybe able to help you

Download it free here -> https://github.com/cameraki/Magento-Magmi-Via-Cron-Extension/

Makes sense I post the code here in case the link goes offline, so here it is. ps, since I made a readme with the extension on github I have added this information at the bottom to help you get started.

app/code/local/Oli/Magmi/Cron.php

<?php
/**
 * Created by PhpStorm.
 * User: Oli
 * Date: 02/11/2017
 * Time: 17:05
 */

/* Change importer to the name of your Magmi directory */
require_once(Mage::getBaseDir() . "/importer/plugins/inc/magmi_datasource.php");
require_once(Mage::getBaseDir() . "/importer/integration/inc/productimport_datapump.php");

class Oli_Magmi_Cron {

    const LOG_DIRECTORY = "oli-extensions.log";

    public function runMagmi()
    {


        try {
            $directory = Mage::getBaseDir('var') . '/import/';
            $magmiCsvFile = $directory . 'magmiUploadReady.csv';

            if(!file_exists($magmiCsvFile)) {
                throw new Exception('CSV file for Magmi to upload cant be found.');
            }
        } catch (Exception $e) {
            $message = date('F jS (l) Y h:i:s A') . " === MAGMI (" . __METHOD__ . ") === Could not load todays magmiupload file. It must not have run today. This script will now stop: ".$e->getMessage();
            Mage::log($message, null, self::LOG_DIRECTORY);
            die();
        }

        $productArray = $this->csvToArray($magmiCsvFile);

        $indexes = 'catalog_product_attribute,catalog_product_price,cataloginventory_stock';
        $mode = 'update';
        $profile = 'cronUpdatePrices';
        try {
            $this->import($productArray, $mode, $profile);
        } catch (Exception $e) {
            $message = date('F jS (l) Y h:i:s A') . " === MAGMI (" . __METHOD__ . ") === There was an issue importing the products: ".$e->getMessage();
            Mage::log($message, null, self::LOG_DIRECTORY);
        }
        try {
            $this->reindex($indexes);
        } catch (Exception $e) {
            $message = date('F jS (l) Y h:i:s A') . " === MAGMI (" . __METHOD__ . ") === There is a problem with the reindex function: ".$e->getMessage();
            Mage::log($message, null, self::LOG_DIRECTORY);
        }
        /* log that it actually ran */
        $message = date('F jS (l) Y h:i:s A') . " === MAGMI (" . __METHOD__ . ") === Has finished running";
        Mage::log($message, null, self::LOG_DIRECTORY);

    }



    public function csvToArray($filename)
    {
        $productArray = array_map('str_getcsv', file($filename));
        array_shift($productArray);
        $newProductArray = array();
        /* Customise the item array index's to match the file you are importing */
        foreach ($productArray as $key => $item) {
            $newProductArray[$key]['sku'] = $item[0];
            $newProductArray[$key]['buy_price'] = $item[1];
            $newProductArray[$key]['price'] = $item[2];
            $newProductArray[$key]['store'] = $item[3];
            $newProductArray[$key]['price_updated'] = $item[4];
            $newProductArray[$key]['amazon_euros'] = $item[5];
            $newProductArray[$key]['amazon_dollars'] = $item[6];
            $newProductArray[$key]['qty'] = $item[7];
            $newProductArray[$key]['is_in_stock'] = $item[8];

        }
        return $newProductArray;
    }



    private function import($items, $mode, $profile)
    {
        if (count($items) > 0) {
            try{

                $dp = new Magmi_ProductImport_DataPump();
                $dp->beginImportSession($profile, $mode);
                foreach ($items as $item) {
                    $dp->ingest($item);
                }
                $dp->endImportSession();

            } catch (Exception $e) {
                $message = date('F jS (l) Y h:i:s A') . " === MAGMI (" . __METHOD__ . ") === Seems to be an issue with the import function: ".$e->getMessage();
                Mage::log($message, null, self::LOG_DIRECTORY);
            }
        }
    }

    private function reindex($string)
    {
        /** @var $indexer Mage_Index_Model_Indexer */
        $indexer = Mage::getModel('index/indexer');

        $processes = array();

        if ($string == 'all') {
            $processes = $indexer->getProcessesCollection();
        } else {
            $codes = explode(',', $string);
            foreach ($codes as $code) {
                $process = $indexer->getProcessByCode(trim($code));
                if ($process) {
                    $processes[] = $process;
                }
            }
        }

        /** @var $process Mage_Index_Model_Process */
        foreach ($processes as $process) {
            $process->reindexEverything();
        }
    }
}

app/code/local/Oli/Magmi/etc/config.xml

<?xml version="1.0"?>
<config>
    <modules>
        <Oli_Magmi>
            <version>0.0.1</version>
        </Oli_Magmi>
    </modules>
    <global>
        <models>
            <oli_magmi>
                <class>Oli_Magmi</class>
            </oli_magmi>
        </models>
    </global>
    <crontab>
        <jobs>
            <update_prices_via_magmi>
                <schedule>
                    <cron_expr>30 3 * * 2</cron_expr>
                </schedule>
                <run>
                    <model>oli_magmi/cron::runMagmi</model>
                </run>
            </update_prices_via_magmi>
        </jobs>
    </crontab>
</config>

app/etc/modules/Oli_Magmi.xml

<?xml version="1.0"?>
<config>
    <modules>
        <Oli_Magmi>
            <active>true</active>
            <codePool>local</codePool>
        </Oli_Magmi>
    </modules>
</config>

Magento Magmi Via Cron Extension

Magento extension to run Magmi via Magento's cron from a CSV file

Note: This only works if you have Magmi (Magento Mass Importer)

What does this do?

Magmi allows you to make changes to products in bulk and fast. Its fast because its a direct import into the database. So use Magmi with caution and backup your database before you import.

This extension allows you to run Magmi automatically using Magento's cron. Great for updating product pricing automatically every day, or changing stock levels in bulk once a week.

Can I use it out the box?

You need to make sure you have Magmi on your server. Then:

  • On line 10 & 11 of app/code/local/Oli/Magmi/Cron.php change the "/importer/...." to the location of your Magmi install. Eg, "/magmi/...."
  • On line 23 of app/code/local/Oli/Magmi/Cron.php change $magmiCsvFil to be the address of the CVS file you are importing from. Once you have changed this, change the lines around line 66 to map the columns from the CSV file into an array. The keys of $newProductArray[$key]['...'] should match the attributes you are changing. Eg, if you want to update stock levels, you will use $newProductArray[$key]['qty'] since 'qty' is what Magento's stock levels are.

Customisation

  • On line 36 of app/code/local/Oli/Magmi/Cron.php this sets which Magento settings you want to re-index. The ones I add are: catalog_product_attribute catalog_product_price cataloginventory_stock You may want to add more or change these.
  • On line 37 of app/code/local/Oli/Magmi/Cron.php this is Magmi mode. "create" creates and updates items, "update" updates only, "xcreate" creates only.
  • On line 38 of app/code/local/Oli/Magmi/Cron.php change this to match the name of the profile you have created in Magmi. You can use 'default' if you want to use the default profile however this is worth looking at because you want to make sure On the Fly indexing is turned off becauase this extension re-index's your magento store at the end. If you use a profile that has On the Fly indexing, it will re-index after each product, taking up a lot of server load for no reason.

More info You can find more information about the Magmi DataDump API here -> http://wiki.magmi.org/index.php/Magmi_Datapump_API

Oli Girling
  • 605
  • 8
  • 14
0

In some scenarios I had to run Magmi from Magento cron job so I had something like this:

/**
 * Includes Magmi files
 */
protected function _includeMagmi()
{
    $magentoBaseDir = Mage::getBaseDir();
    require_once($magentoBaseDir . DS . 'magmi' . DS . 'inc' . DS . 'magmi_defs.php');
    require_once($magentoBaseDir . DS . 'magmi' . DS . 'integration' . DS . 'inc' . DS . 'magmi_datapump.php');
}

/**
 * Runs Magmi to update/create for specified data
 *
 * @param string $profile Magmi import profile (@see magmi/conf/
 * @param string $mode
 * @param array $data
 */
protected function _runMagmiImport($profile, $mode, array $data)
{
    $this->_includeMagmi();

    $logger = Mage::getModel('mario/magmi_logger');

    /** @var Magmi_ProductImport_DataPump $dataPump */
    $dataPump = Magmi_DataPumpFactory::getDataPumpInstance('productimport');
    $dataPump->beginImportSession($profile, $mode, $logger);
    foreach ($data as $update) {
        $dataPump->ingest($update);
    }
    $dataPump->endImportSession();

    return $logger;
}
Mario
  • 502
  • 4
  • 4