23

I've frequently run into an annoyance in Emacs's sql-mysql mode, and I'm wondering if anyone has a solution or better workaround. Anytime I try to send a query from an sql-mode buffer to an active SQL process buffer, that query cannot be larger than 4k. If it's larger than 4k, it appears that some sort of break - perhaps a newline - is inserted, and this causes the mysql interpreter to throw an error on the following line.

sql-mysql is implemented by sql.el, and uses the function sql-send-region to send query regions (or whole buffers) to the selected SQL process buffer. sql-send-region calls comint-send-region, which in turn calls process-send-region. process-send-region is a C function that calls send_process, both in src/process.c in the Emacs source.

It looks like this may just be a limitation produced by the 4k buffer on an IPC pipe. Since it appears that kernel hacking is necessary to change this size, that's not a great answer.

What I guess I'm puzzled by is why the SQL sent through the pipe is not properly reassembled by the mysql client if it's larger than 4k. Any ideas?

Emacs version: GNU Emacs 23.3.1 (x86_64-pc-linux-gnu, GTK+ Version 2.24.10) of 2012-03-25 on allspice, modified by Debian

mysql -V: mysql Ver 14.14 Distrib 5.5.24, for debian-linux-gnu (x86_64) using readline 6.2

Sql Mysql Options: -A -C -n (NB I've tried both with and without -n (unbuffered) and neither fixed this issue)

Yamaneko
  • 3,433
  • 2
  • 38
  • 57
Ryan M
  • 688
  • 6
  • 12
  • BTW, the best workaround I've so far found is to source the query directly from the saved file, i.e. "\. sql_long_query.sql". However, this is rather annoying when trying to debug portions of a query or multi-query files. That's why I'd like to be able to use the sql-send-region method even for regions larger than 4k chars. – Ryan M Aug 14 '12 at 15:26
  • Also note that simply copying and pasting a query of longer than 4k into the sql-mysql buffer encounters the same issue. More evidence that the problem is at or below the send_process C func level. – Ryan M Aug 14 '12 at 15:28
  • can you try running emacs from rlwrap ? – ramrunner Aug 28 '12 at 20:33
  • @ramrunner, that would require emacs to be run from a terminal, which isn't always (practically never, imo) desirable. – Squidly Aug 29 '12 at 12:51
  • @MrBones: i don't get it. couldn't you change your "button" in the gui to execute "rlwrap emacs"? that's trivial. – ramrunner Aug 29 '12 at 13:28
  • Sorry, I didn't mean run from a terminal, I meant run inside a terminal. I don't like using the curses interface to emacs. – Squidly Aug 29 '12 at 13:32
  • Hmm. I wouldn't want to try rlwrap outside of emacs. It might work as a wrapper around mysql inside comint... maybe. – Ryan M Aug 30 '12 at 15:53
  • I wrote a tiny script to wrap mysql calls in rlwrap. Works great from shell. No such luck from within emacs. I keep getting: rlwrap: Oops, crashed (caught SIGFPE) - this should not have happened! – Ryan M Aug 31 '12 at 15:40
  • I tried set-buffer-process-coding-system, but none of the settings I tried made any difference. Still stuck. – Ryan M Sep 10 '12 at 13:12

1 Answers1

5

I suspect the culprit to be Emacs's process communication code, used by comint, allocating PTYs to talk to processes. While these are useful for interactive work because they allow job control, they are also limited in how much data they can transfer in one chunk without an intervening newline. Emacs can also be told to use pipes, which do not have this limitation.

To test this, start a fresh Emacs, evaluate M-: (setq process-connection-type nil), and start sql-mysql. If the problem goes away, this is your culprit. In that case, you will want to use something like:

(add-hook 'sql-mysql-mode-hook
          (lambda ()
            (set (make-local-variable 'process-connection-type) nil)))

to make sure that process-connection-type gets reset only in MySQL interaction buffers.

EDIT

According to http://tinyurl.com/car439o/, Emacs no longer bothers to interrupt long PTY output with newline+EOF pairs. Although the commit is from 2010-04-13, it only appeared in Emacs 24, released in 2012. This would explain why the problem is apparently not reproducible in 24.2.1. If you are using Emacs prior to 24, try upgrading.

user4815162342
  • 141,790
  • 18
  • 296
  • 355
  • Okay, this is pretty close to a complete answer. M-: (setq process-connection-type nil) does clear up the 4k limit on incoming SQL. As an unfortunate side effect, it also seems to suppress the MySQL prompts, probably because the MySQL client detects being called from a pipe and changes its defaults accordingly. It also causes the MySQL client to exit on errors, rather than report an interactive error message. Adding the -n flag to the mysql call (via customize) is necessary, and -t will restore the tabular output format (-s mode is the default on pipes.) – Ryan M Sep 10 '12 at 21:45
  • Is there any way to change the behavior of the PTY to avoid inserting newlines, or to increase its buffer size to something larger? – Ryan M Sep 10 '12 at 21:45
  • It looks like turning off ICANON on the PTY might suppress it sending a newline at the end of each buffer's worth of text. How might we go about clearing the ICANON flag on the PTY when allocating it from within Emacs? – Ryan M Sep 10 '12 at 21:55
  • Just FYI, the reason I haven't awarded the bounty is the quit on error behavior of the pipe-connected mysql. This is a cure that is worse than the original problem, which is to be able to do development of large SQL queries more conveniently. Having to restart mysql on every error is even less convenient than having to enter \. into the sql-mysql buffer. We're closer, but not close enough to accept the answer yet. – Ryan M Sep 10 '12 at 21:58
  • OK, my answer was only a starting point for investigation. It could be that Emacs is being overcautious with its limitation on how much it sends to a PTY in a single bunch. I'll try to dig up more information on this tomorrow. It might also be possible to somehow force `mysql` into interactive mode. – user4815162342 Sep 10 '12 at 22:34
  • 1
    Interestingly, I cannot repeat the problem you're having with, say, `zsh` in shell-mode using PTYs. I use `C-u 100000 a RET` to send a command of 100k length, and (after waiting for a while for Emacs to parse the output), the "command aaaaaaaaa... too long" error that zsh produces has exactly 100k a's in the error message. If zsh is able to receive correct output from Emacs, why isn't mysql? – user4815162342 Sep 10 '12 at 22:37
  • That's an interesting question, and one that I tried to work around using set-buffer-process-coding-system. None of the coding systems seemed to help, though. – Ryan M Sep 11 '12 at 14:31
  • Coding systems define how the buffer characters are encoded to bytes to send over the wire, they don't affect internal communication limits. – user4815162342 Sep 11 '12 at 18:21
  • @RyanM Please take a look at the edit; which version of Emacs are you using? – user4815162342 Sep 11 '12 at 20:00
  • let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/16566/discussion-between-ryan-m-and-user4815162342) – Ryan M Sep 11 '12 at 21:07