2

I am trying to compile ATLAS with MinGW. I started to solve problems, but now I'm stuck with a simple one: the Makefile of ATLAS tries to probe OS with an uname.exe which is provided with MinGW.

If I run MinGW shell (sh.exe) I can call uname. If I open a Windows Command Prompt, I could call uname (so the %PATH% variable and $PATH variable are set up properly).

The code calls internally something like system("uname -s >\tempfilneame &2>1"). The main problem is that (as I analyzed FileMon's output) that the application (sh.exe) create a cmd.exe process, and than execute my command, but uname is not found. If I add system("d:/.../bin/uname -s >\tempfilename &2>1") to the code everything works fine. So it should be a problem with PATH variables. If it's not neccesary I wouldn't modify every single call in the code. The question is: what is special about the cmd.exe process started, so it doesn't recognize uname, and how could I solve this issue.

WebMonster
  • 2,981
  • 2
  • 22
  • 29

1 Answers1

2

The purpose of Msys is to provide a minimal quasi-"*nix" (i.e., non-POSIX) Bourne Shell (a "*nix" command line interpreter) in which to execute programs, commands and shell scripts as if on a "*nix" system (refer to http://www.mingw.org/). If, for whatever reason, one wishes to run MinGW/bin and Msys/1.0/bin executables from the MS Windows command line interpreter (cmd.exe), the MS Windows environment %PATH% variable must include the paths to these two "bin" subdirectories -- such as ".;C:\MinGW\bin;C:\MinGW\msys\1.0\bin;%PATH%".

The following information refers strictly to building ATLAS libraries from within Msys sh.exe invoked with msys.bat using the GCC 4.6.2 compiler suite provided by MinGW.

Make sure your Msys $PATH environment variable includes ".:path to your MinGW and Msys bin directories" -- i.e., ".:/C/MinGW/bin:/C/MinGW/msys/1.0/bin:$PATH" if that's where you installed MinGW and Msys (this should be done by the /etc/profile script when a sh login occurs, as by the \msys\1.0\msys.bat file).

The following changes must be made to any ATLAS program statement which invokes a shell command or executable (other than "make") using the system() function:

1) Change "./executable" to ".\\executable" or "executable" since the MS Windows cmd.exe shell (or more correctly, the called MS Windows system library routines) interprets "./executable" as the executable '.' with a command option '/executable', thus the error message: "'.' is not recognized as an internal or external command, operable program or batch file." Make sure to invoke ATLAS/configure with "-v 2" in order to locate any offending system() calls -- there are only a few!!!

ATLAS/bin/atlas_install.c
ATLAS/CONFIG/src/atlbench.c
ATLAS/CONFIG/src/config.c
ATLAS/tune/blas/gemm/userindex.c
ATLAS/tune/blas/ger/r1search.c
ATLAS/tune/sysinfo/emit_buildinfo.c

2) Change "command ; command" to "command && command" since MS Windows cmd.exe does not accept compound command statements using the ';' separator as in other "*nix" shells or "make".

3) Replace any occurrence of single quotes with double quotes delimiting string format specifiers in sprintf() statements for commands passed to system() calls; especially in the SpewItForth function of the ATLAS/CONFIG/src/config.c file. For example, change "... '%s' ..." to "... \"%s\" ...". This is not required for "make" commands invoked with system() calls. Since the MS Windows cmd.exe shell expects CRLF end-of-line termination, the following changes must be made to the ATLAS/bin/atlas_install.c and ATLAS/tune/sysinfo/emit_buildinfo.c files.

In ATLAS/bin/atlas_install.c, line 133 from:

    if ( (sp == NULL) || (str[0] == '\0') || (str[0] == '\n') )

to:

    if ( (sp == NULL) || (str[0] == '\0') || (str[0] == '\r') || (str[0] == '\n') )

line 165, from:

    else if (ln[0] == '\0' || ln[0] == '\n') ch=def;

to:

    else if (ln[0] == '\0' || ln[0] == '\r' || ln[0] == '\n') ch=def;

line 175, from:

    else if (ln[0] == '\0' || ln[0] == '\n') ch=def;

to:

    else if (ln[0] == '\0' || ln[0] == '\r' || ln[0] == '\n') ch=def;

In ATLAS/tune/sysinfo/emit_buildinfo.c, line 66 from:

    for (i=0; ln[i]; i++) if (ln[i] == '"' || ln[i] == '`') ln[i] = '\'';

to:

    for (i=0; ln[i]; i++) if (ln[i] == '"' || ln[i] == '`') ln[i] = '\"';

line 68 from:

    for (i--; i >= 0 && (ln[i] == ' ' || ln[i] == '\n' || ln[i] == '\t'); i--);

to:

    for (i--; i >= 0 && (ln[i] == ' ' || ln[i] == '\r' || ln[i] == '\n' || ln[i] == '\t'); i--);

Also, in the ATLAS/tune/sysinfo/emit_buildinfo.c file, change:

  if (CommandOneLine("date",DATE)) strcpy(DATE, "UNKNOWN");

to:

  if (CommandOneLine("date /t",DATE)) strcpy(DATE, "UNKNOWN");

since the MS Windows cmd.exe "date" command will prompt "Enter the new date: (mm-dd-yy)" and wait for a response which will never be entered.

Although I made the above changes to each affected file, adherence to basic software engineering and configuration management principles would dictate consolidation of duplicate code into single procedures wrapped with some form of "#if defined(ATL_OS_WinNT) && defined(__MINGW32__)" processing directive to keep the changes from adversely impacting installation of ATLAS on other "*nix" operating systems. In this regard, I would not recommend using a "patch" approach to incorporate MinGW32 specific changes to an ATLAS distribution, but merge the requisite changes into the latest ATLAS development version. Of course, "patching" may be the only alternative to fixing unsupported ATLAS releases.

Over the past few days I was able to build tuned ATLAS 3.8.4 static libraries with MinGW, Msys and the GCC 4.6.2 compiler suite downloaded from:

http://sourceforge.net/projects/mingw/files/

and installed with mingw-get-inst-20111118.exe on an 11 year old Dell Workstation 530 with dual Intel Xeon (Pentium 4 with MMX, SSE and SSE2 [L1 cache 8 KBytes, L2 cache 256 KBytes], but no Hyperthreading) 1.5 GHz processors, 400 MHz FSB, and 1024 MBytes RDRAM running Windows 2000 Professional SP4.

The above modifications are not comprehensive, but should get you further along. I am planning to document all my changes to the ATLAS 3.8.4 distribution, and post it on the "ATLAS on Windows" forum at http://sourceforge.net/projects/math-atlas/forums/ after I verify my tuned configuration and do a complete shared library build using LAPACK-3.4.0.

I hope this information will be helpful to you and others attempting an ATLAS build with MinGW and without Cygwin, on a Windows platform.

SPECIAL NOTE:

Calls to system() with commands directing output to a temporary file name obtained from a call to tmpnam() will result in temporary files being created in the root directory of the current drive. The temporary file names are prefixed with '\\', which is defined in MinGW/include/stdio.h (see P_tmpdir).

IMPORTANT ADDENDUM:

(A) Make sure to use full path names when specifying the source and build directories using the '-s' and '-b' command options for ATLAS/configure. In my case, the following sh script shows how I invoked ATLAS/configure from ATLAS_build:

#!/bin/sh

SRCDIR=/g/Progs/MinGW/msys/1.0/home/GaryD/temp/ATLAS
BLDDIR=/g/Progs/MinGW/msys/1.0/home/GaryD/temp/ATLAS_build
LAPACK=/g/Progs/MinGW/lib/gcc/mingw32/4.6.2/liblapack.dll.a

CC=gcc
C_DEFS="-D c -DL2SIZE=262144"  # Not used, set using -f 256
F_DEFS="-D f ''"               # Not used, let configure set these
C_FLGS="--cc=$CC --with-netlib-lapack=$LAPACK"
D_FLGS="-d s $SRCDIR -d b $BLDDIR"
A_FLGS="-O 8 -s 1 -A 20 -V 8 -b 32 -f 256 -t 2 -m 1495"
S_FLGS="-Si bozol1 1 -Si archdef 0 -Si cputhrchk 0"

$SRCDIR/configure $C_FLGS -v 2 $D_FLGS $A_FLGS $S_FLGS

(B) I believe the preferred build directory is assumed to be a subdirectory under the ATLAS source directory, such as ATLAS/ATLAS_build, since a file ../../CONFIG/error.txt cannot possibly be located by ATLAS/bin/atlas_install.c if the source and build directories are at the same level.

(C) Do not make the mistake I did in assuming the '-m ####' configure option implicitly creates a '-D c DPentiumCPS=####' configure flag. You may need to explicitly pass '-D c -DPentiumCPS=####' or '-D c -DWALL' as documented in section 3.4, "Changing the way ATLAS does timings", of the "ATLAS Installation Guide" which can be viewed here:

http://math-atlas.sourceforge.net/atlas_install/atlas_install.html#SECTION00044000000000000000

By not using a cycle-accurate wall timer, results from sanity tests, such as those from "xsslvtst.exe -n 167 -r 83 -O 2 c r -U 2 u l", resembled this:

    ORD  UPLO       N    NRHS     lda     ldb       TIME     MFLOP        RESID
    ===  ====  ======  ======  ======  ======  =========  ========  ===========

      C     U     167      83     167     167      0.015    257.82  1.762022e-003
      C     L     167      83     167     167      0.016    241.70  1.870298e-003
      R     U     167      83     167     167      0.000 2229332063710638100.00  1.870298e-003
      R     L     167      83     167     167      0.000 2229332063710638100.00  1.762022e-003

    4 TESTS RUN, ALL PASSED.

Each invocation of the test produced different spurious values for MFLOP, possibly resulting from arithmetic division by a very, very, very small non-zero value. After rebuilding ATLAS using '-D c -DWALL', these MFLOP results (and no doubt other timing measures) were rectified.

(D) Line 35 of ATLAS/CONFIG/src/probe_OS.c needs a check for MinGW as this:

else if(strstr(ln, "WIN") || strstr(ln, "MINGW"))

and insert the following after the "IRunArchInfo_linux: xarchinfo_linux" block in ATLAS/CONFIG/src/Makefile (just cut, paste and edit the IRunArchInfo_linux block in the Makefile):

IRunArchInfo_winnt: xarchinfo_x86
    - rm -f config0.out
    $(MAKE) $(atlrun) atldir=$(mydir) exe=xarchinfo_x86 args="$(args)" \
            redir=config0.out
    - cat config0.out

(E) The gcc-4 lines may need to be changed to just gcc in the ATLAS/CONFIG/src/atlconf.txt file since the MinGW gcc executable is just gcc.exe and not gcc-4.exe for this latest version of the MinGW32 gcc compiler suite. The use of configure flags to override compiler selections and to change/append compilation flags is probably preferred over editing the atlconf.txt file (refer to section 3.2 of the "ATLAS Installation Guide") by the ATLAS developers; but I was impatient.

(F) If you specify SSE2 with '-V 8' ATLAS/configure option as for my architecture, the following change must be made to the ATLAS/include/atlas_asm.h file.

Line 133 from:

    #if defined(ATL_SSE1) && !defined(ATL_3DNow)

to:

    #if (defined(ATL_SSE1) || defined(ATL_SSE2)) && !defined(ATL_3DNow)