urxvtd performance problems and possible fixes

[Marc Lehmann]

On Fri, Dec 07, 2007 at 01:42:31PM +0000, Reuben Thomas <rrt at sc3d.org> wrote:
> >You need to use an idle watcher.
> 
> I don't understand. Won't that simply postpone the code?

Yes.

> I don't mind it running (just like other apps I have open) I just don't
> want it blocking the other terminals.

You can't do that without an idle watcher - it runs when the app has nothing
better to do, which means it has to use a lower priority than the tasks that
update the other terminals.

> >Doing it in the background will also *decrease* performance, so wouldn't
> >fix any performance problem.
> 
> I don't mind decreasing performance so long as I can use my terminals! It's 
> having to wait until the reformatting completes that annoys me.

Well, it will not be implemented, as I cannot imagine that being the problem.

> 1GHz/512Mb, and the memory is not overloaded: I typically have ~100Mb used 
> as cache (aka free). I'm only using 40,000 save lines.

Are you sure you are looking at it right? Free certainly is a different
value than cache on all opertaing systems. Cache means the memory isn't
free, but used for other purposes. It doesn't preclude the scrollback
being swapped out, for example.

On thet machine, there is no way that, under normal conditions, rewrapping
40000 lines would take minutes. Not even if you have 10 windows open will
it take that long, and not even on a 1ghz machine.

> >The solution would be to use not urxvtd but separate terminal windows,
> >then resizing a few will not inhibit all of them.
> 
> But then wouldn't I lose the memory savings?

A bit of it, depending on a lot of factors. It can easily be a few hundred
kb per process.

> In any case, with my window manager, the typical case is that all
> windows on screen are resized at the same time.

Thats fine, but there is really no way it could take *minutes*. A few
seconds I could accept, but minutes is out of the question. There must be
other factors around, like urxvtd running under valgrind, or beign swapped
out etc :)

> I have maybe 2 or 3 such terminals on average. I don't want them fast, I 
> just want them not to be locked up for several minutes.

Then it cannot be the wrapping.

> With the search I'd like a) to be able to use other terminals while one is 
> busy searching (though why it takes significant time to search 40,000 lines 
> is a mystery to me; Emacs doesn't) and b) to be able to interrupt it.

The regexes you can use are quite a bit more powerful then the ones emacs
offers, which comes at a price. Also, being written in perl and using a
straight conversion between internal memory format and utf-8 also doesn't
help the speed.

In any case, I want that too, patches are welcome :->

> It does seem to access the disc a lot at this point, sure. Indeed I noticed 

Thats your problem, not enough memory.

> Mem:    507792k total,   498888k used,     8904k free,    25852k buffers
> Swap:   522104k total,   261052k used,   261052k free,    67636k cached
> 
> i.e. ~100Mb of RAM free.

I see 25mb, not 100, which is rather little, given that linux (I assuem
you use linux) will not use all the buffer cache.

In any case, that is your problem.

> As far as I can see, the performance, given the amount of memory involved, 
> is not unreasonable.

Yes, if programs get swapped out, the performance will go down considerable.
"Moving into background" will singificantly help, as the program does not run
when it needs memory to be swapped in. The solution is not to swap it out
(for example using memlock) or reduce its memory requirements.

If you compile without --enable-unicode3, then each character will need 6
bytes instead if 8, which helps a lot with big scrollbacks.

Otherwise, keep in mind that big scorllbakc buffers use a lot of ram. Its
still much less than xterm or other terminal emulators use, though :->

> So, I'm not complaining about that. The main thing I 
> would like to see improved is the interactiveness of urxvt when under load. 

Then avoid swapping. If your program starts thrashing then performance
will be gone, nothing to be done about that.

> myself, so the simpler expedients of being able to turn off reformatting 
> and interrupt Perl searching would keep me happy.

Reformatting is fast, the problem you have is thrashing, which indeed
destroys performance completely. But programs can not do anything about that
except being more parsimounous on memory, which urxvt tries really hard.

(In fact, one of the todo items on my list is in-memory compression of
scrollback, but that might not ever be implemented).

> (For comparison, Emacs, into which I typically have far more text loaded 
> (maybe 100 buffers at the moment) hardly ever has these sorts of problems. 
> Unfortunately, its terminal mode is not as good as urxvt.)

You probably use it a lot more, causing urxvt to be swapped out and emacs to
be in. Or something like that. There is only so much you can do when memory
runs out. I am sorry, but thats it, really. You could try rxvt, which, I
think, uses 5 bytes/character, but it doesn't get much better.

-- 
                The choice of a       Deliantra, the free code+content MORPG
      -----==-     _GNU_              http://www.deliantra.net
      ----==-- _       generation
      ---==---(_)__  __ ____  __      Marc Lehmann
      --==---/ / _ \/ // /\ \/ /      pcg at goof.com
      -=====/_/_//_/\_,_/ /_/\_\