Why does Cygwin execute shell commands very slowly?

Question

Just tests a very simple command like:

while true; do bash -c "echo hello"; done

You will find how much slow the bash in Cygwin is. Does anybody know why?

It's a fresh install of cygwin 1.7 on win7.

thanks to Jared's testing idea, I modified the command to this(adds bash -c):

time for i in {1..10} ; do bash -c "echo Hello" ; done
Hello
...
real 0m7.711s //it's the problem
user 0m0.091s
sys 0m0.476s

Maybe the trouble is simply that Windows 7 can't create processes very fast? — Jonathan Leffler, Jul 25 '11 at 03:28
Windows can't *fork* processes very fast. In fact, it doesn't `fork` at all. Cygwin fudges it, but can't do it anywhere near as efficiently as a real *nix can. Each process is starting from scratch, and has to load and init the cygwin DLL, libc, etc that a fork-and-go system already has loaded. — cHao, Jul 25 '11 at 04:31
But the strange is that during the command creating the 10 processes for about 7.7 seconds, the CPU is empty and you can see that the user & sys time is slightly. It looks like bash is waiting for something, but what does it waiting for? — Lcsky, Jul 25 '11 at 05:14
I get 0.970s real, 0.090s user, 0.307s sys. (I thought it might be a startup script, but `bash -c ...` doesn't invoke any startup scripts.) `strace -o bash.strace bash -c "echo hello"` might tell you something useful. — Keith Thompson, Aug 02 '11 at 00:42
`> time ls -la ~/c/cygwin64/bin` gave me an unimaginable 3m46.373s!! It was always slow, but recently our IT dept. changed the Windows login domain and usernames, and since then all hell broke lose... — ysap, Sep 09 '16 at 16:32
Might be some software interfering, check this out: https://stackoverflow.com/a/44158105/3437245 — user3437245, May 24 '17 at 12:26

score 16 · Answer 1 · answered Mar 07 '14 at 10:57

16

How about excluding Cygwin paths from your antivirus software ?

answered Mar 07 '14 at 10:57

dotz

884
1
8
17

Which paths are cygwin paths ? I excluded this one - C:\cygwin64. Btw, for some antiviruses such as norton, you can find the "exceptions" option somewhere and add this folder to the exceptions list. – Steam Jan 01 '15 at 00:22
Should I also add the folder to which cygwin downloads all its packages ? That is, the one with the weird name like - `ftp%3a%2f%2fftp.harvard.univ.edu%2fpub%2fcygwin%2f` – Steam Jan 01 '15 at 00:29
As far as I know, the downloaded packages folder is used only for caching downloaded packages and installatin/reinstallation with Cygwin setup. Those files are not being used when running cygwin. I think that c:\cygwin64 should be sufficient. I'd check Windows system folders for cygwin DLLs too. Not that I'd suggest adding vital Windows folders to AV exceptions :) – dotz Jan 01 '15 at 09:59
I use Avast antivirus. Cygwin is excruciatingly slow to do any operation. Disabling the Avast shield makes Cygwin very fast. – Florin Andrei Mar 24 '16 at 04:09
In my case, I had a really poor performing PS1 (custom prompt) variable set. Once I removed it, everything was really fast. – Derek Greer May 22 '18 at 14:36
Wow I ran cygwin ./configure on some project and already took half an hour. Found this answer, and I had bitdefender running, add exception to it and it finished like 1 sec :/. BRAH... – Wappenull Oct 13 '19 at 13:37
@Wappenull, configure does caching – dotz Feb 18 '20 at 03:26
**Unsafe** and **unnecessary!** … Antivirus programs cache known files to prevent needless rescanning and avoid performance drain. Cygwin is great—I use it every day—but there are security risks in any system that comprises executables and scripts that are user-writable and unsigned. If the Cygwin repository maintainers missed an infected update, you would be infected and unprotected by your antivirus. Even if the script is not malicious, it could be vulnerable to attacks using compromised external servers or malicious data files. – Alex Quinn Aug 05 '20 at 15:54
@AlexQuinn Windows version of PyCharm suggest adding source directories of Python projects to Windows Defender excluded paths exactly because of better performance, possibly basing on the fact that you could scan for malware on many earlier steps (download, unzip). Same for Cygwin - files get downloaded from the web, so they could be scanned by the AV software at that point, then they could be left as they are. So, TBH I'm not exactly sure if your advice is valid and I'm not exactly sure what is the threat level. – dotz Nov 20 '20 at 16:36
@dotz PyCharm advice is a different issue. Code that you are actively developing changes a lot, so antivirus (AV) caching can't save you. Once an AV has scanned a file, it won't be scanned again, unless the AV is updated or reconfigured. Needless AV exclusions make it easier for sophisticated viruses to evade your defenses. – Alex Quinn Nov 22 '20 at 09:25

metatechbe · Answer 2 · 2018-08-18T18:59:48.090

7

Below are 3 possible causes which I encountered. Of course, other problems not mentioned can also cause slow shell commands in Cygwin.

If you have the option "Automatically detect settings" in "LAN Settings", Windows will use the WPAD protocol to discover the local HTTP proxy. First it will send a DHCP "Inform" request with option 252, then it will try a DNS lookup on "wpad". These 2 operations can take a few seconds to time-out.
If the shell accesses some paths like /cygdrive/... , a NetBIOS name query will be executed, which can also take some time to time out.
If the shell accesses some paths like //mypath/... , a NetBIOS name query will be executed, which can also take some time to time out.

Solutions :

Disable "Automatically detect settings" in "LAN Settings" in the Windows "Internet Options" control panel.
Add the following entry in %SystemRoot%\system32\drivers\etc\hosts :

127.0.0.1 localhost cygdrive wpad

Make sure to avoid double slashes at the beginning of all paths.

edited Aug 18 '18 at 18:59

answered Sep 17 '12 at 19:40

metatechbe

657
9
9

"Disable "Automatically detect settings" in "LAN Settings"." - How do I do this ? I can't find the cygwin installer. – Steam Jan 01 '15 at 00:34
1

That is not a Cygwin setting, but rather a Windows setting – Demi Jan 10 '15 at 02:44
1

@Steam, search for *Internet Options* from Start > Search, open it, click Connections tab, click Settings, and uncheck Automatically detect settings – SomeGuyOnAComputer Jul 28 '16 at 20:33
Right on. Thanks man, been looking for a while for a solution to slow directory changes on network UNC paths. To be a bit clearer, if you are browsing \\somehost\some\path, then somehost should be in the hosts file. This is especially helpful when using DFS mapped UNC paths. – Andrew Taylor Oct 18 '18 at 20:53

score 7 · Answer 3 · answered Jul 25 '11 at 03:18

7

Check your path. Referring a non-existant path or a very slow network share can cause the symptoms you are describing.

answered Jul 25 '11 at 03:18

Jeremy J Starcher

23,369
6
54
74

score 6 · Accepted Answer · answered Aug 22 '11 at 04:43

6

Finally, I found the source - a service named "QQPCMgr RTP Service" running on my office computer, it's the real time protection service of "QQ PC Manager".

By disabling it, the time of the script in the question falls back to:

real    0m0.943s
user    0m0.105s
sys     0m0.231s

I have told the developers of QQPCMgr about this, hope they will find the reason.

This still much slower than Linux, but gets the same "real time" of other cygwin computers.

Thank you all!

answered Aug 22 '11 at 04:43

Lcsky

825
1
8
17

7

How did you determine this? I looked for the service, but don't see it and I'm having the same issue. time for i in {1..10} ; do bash -c "echo Hello" ; done Hello ... real 0m18.070s user 0m0.061s sys 0m0.385s – Robert Swisher Apr 17 '13 at 22:13

score 2 · Answer 5 · answered Sep 05 '17 at 09:51

This is how this problem manifested for me. For the first say 20 commands entered at the cygwin prompt it is fast, then it abruptly becomes painfully slow.

I checked that every item in my path was a valid directory.

I tried an authentication fix suggested elsewhere - https://superuser.com/questions/877051/cygwin-from-windows-very-slow-laggy/1247517 . It didn't work for me.

I replaced my exisitng virus scanner (System Center Endpoint Protection) with Sophos. That fixed the problem for me. Hope this helps.

score 1 · Answer 6 · answered Mar 20 '13 at 18:35

1

I was able to fix this problem by uninstalling bash-completion and switching from Kaspersky to Windows Security Essentials. (I diagnosed the Kaspersky interference using Process Explorer while running the "echo Hello" benchmark.) Brought my benchmark time from 7 seconds to 0.2.

answered Mar 20 '13 at 18:35

In my case stopping the service AVP (Kaspersky Endpoint Security Service) cut the process startup time from 200ms to almost <1ms. Unfortunately [Microsoft Security Essentials apparently does not protect you](http://www.pcpro.co.uk/reviews/software/379954/microsoft-security-essentials), so it isn't a choice for me – sinelaw Oct 24 '13 at 14:25

score 1 · Answer 7 · answered Dec 28 '16 at 15:46

1

On Comodo Internet Security Premium 10, i added "C:\Users\\Documents\MobaXterm\slash\bin" to the list of trusted files. (Settings -> File rating -> File list). Local terminal is fast again now.

answered Dec 28 '16 at 15:46

Future

388
8
21

score 1 · Answer 8 · answered Jan 08 '20 at 23:21

For Windows Defender:

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows Defender]
"DisableAntiSpyware"=dword:00000001

[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows Defender\Real-Time Protection]
"DisableBehaviorMonitoring"=dword:00000001
"DisableOnAccessProtection"=dword:00000001
"DisableScanOnRealtimeEnable"=dword:00000001

or less dramatically:

powershell -Command Add-MpPreference -ExclusionPath "C:\opt\cygwin64"
powershell -Command Add-MpPreference -ExclusionPath "C:\home"
powershell -Command Add-MpPreference -ExclusionProcess "git.exe"
powershell -Command Add-MpPreference -ExclusionExtension ".c"
powershell -Command Add-MpPreference -ExclusionExtension ".cpp"
powershell -Command Add-MpPreference -ExclusionExtension ".cxx"

For Sophos:

REG ADD "HKLM\SYSTEM\CurrentControlSet\Services\Sophos Endpoint Defense\Scanning\Config" /v "OnAccessExcludeFilePaths" /t REG_MULTI_SZ /d "C:\Program Files (x86)\Sophos\Sophos Anti-Virus\\0C:\home\\0D:\Video\\" /f
REG ADD "HKLM\SYSTEM\CurrentControlSet\Services\Sophos Endpoint Defense\Scanning\Config" /v "OnAccessExcludeProcessPaths" /t REG_MULTI_SZ /d "C:\opt\cygwin64\\" /f

Jared · Answer 9 · 2011-08-12T14:46:51.917

1

Let's put some numbers to it. I ran the following:

time for i in {1..1000} ; do echo "Hello" ; done

The result I get from a standard Cygwin bash window is:

...
Hello
Hello

real    0m0.584s
user    0m0.031s
sys     0m0.000s

And from a xterm bash window on the same system I get:

...
Hello
Hello

real    0m0.037s
user    0m0.016s
sys     0m0.000s

I think this pretty much answers the question for you. The problem is you're going through a "Windows" "cmd" like window, which is inherently slow. Cygwin itself isn't the problem, it's the display trying to keep up that is slowing things down (for this test).

edited Aug 12 '11 at 14:46

answered Jul 25 '11 at 03:35

Jared

1,887
3
20
45

`code` $ time for i in {1..10} ; do bash -c "echo Hello" ; done Hello ... real 0m7.711s user 0m0.091s sys 0m0.476s – Lcsky Jul 25 '11 at 04:07
OK, something is definitely wrong with your setup if it's taking 700ms to execute a single echo. Did you try it in xterm? You have to download the X packages too to get that. – Jared Jul 25 '11 at 18:27
2

You're doing `echo` in the current shell whereas the op is spawning a new shell to do each echo. That's going to have a performance difference. – Burhan Ali Nov 15 '12 at 19:24

score 1 · Answer 10 · answered Jul 25 '11 at 06:13

I voted up on James McLeod, because starting a bash process takes some time, but it doesn't mean it will run commands slower than in UNIX.

Invoking bash -c from within a bash script is near to senseless, and Makefiles can call a lot of bash subprocesses, unless you append ; \ at the end of the commands.

For example, if a Makefile has the following:

echo Hello World
echo Good Bye

It will call two bash processes. To make it faster and call just one bash process:

echo Hello World; \
echo Good Bye

Debian has adopted dash instead of bash as the main shell, because starting many init scripts using bash will make the system take much longer to boot (each script call its own bash process).

score 0 · Answer 11 · answered Jul 25 '11 at 03:14

0

Invoking bash from a shell script can't be fast. Is it faster to just use

while true; echo hello; done

?

answered Jul 25 '11 at 03:14

James McLeod

2,381
1
17
19

5

yeah, it's the reason why I added "bash -c" in the question line, if not, it runs very quickly! But the configure scripts, Makefiles run by invoking a new bash, and so it's very slowly. – Lcsky Jul 25 '11 at 04:05

Why does Cygwin execute shell commands very slowly?

11 Answers11

Linked