Bugzilla – Bug 141135
rsync fails with glibc problem
Last modified: 2006-11-02 14:15:15 UTC
This a dual opteron246 64 bit box using the 64 bit install of suse 10.0. Kernel is the stock 2.6.13-15.7-smp provided by the OS. rsync would fail with *** glibc detected *** free(): invalid pointer: 0x00002aaaab11d4a0 *** when I tried to do a backup using the script shown below. I removed the rsync package using yast and downloaded and compiled rsync from scratch (http://rsync.samba.org/ftp/rsync/rsync-2.6.6.tar.gz). That worked fine. To reproduce the error, I then removed the downloaded rsync as shown below, used yast to install and update rsync, then tried the backup again, and get the error as shown below. mc8:/usr/local/mis/dns/bin # which rsync /usr/local/bin/rsync mc8:/usr/local/mis/dns/bin # rm /usr/local/bin/rsync mc8:/usr/local/mis/dns/bin # yast mc8:/usr/local/mis/dns/bin # rq rsync rsync-2.6.6-2.2 mc8:/usr/local/mis/dns/bin # cat `which backup.spoon.daily ` #!/bin/bash rsync -zav --bwlimit=100 --delete --exclude-from=/usr/local/mis/etc/rsync.exclude --stats / root@10.0.1.20::daily.`hostname -f` mc8:/usr/local/mis/dns/bin # backup.spoon.daily building file list ... *** glibc detected *** free(): invalid pointer: 0x00002aaaab11d4a0 *** /usr/local/mis/sbin/backup.spoon.daily: line 2: 15018 Aborted rsync -zav --bwlimit=100 --delete --exclude-from=/usr/local/mis/etc/rsync.exclude --stats / root@10.0.1.20::daily.`hostname -f` mc8:/usr/local/mis/dns/bin # I got a similar error when using nano, so to fix that I removed the packaged nano and installed from source.
I just wonder about /usr/_local_/bin/rsync - even if you deleted it. Please show the output of which rsync again. Also check if the rsync package is broken using rpm -V rsync (no output is good output ;-) Do you have packages from older installations? Please check using rpm -qa --queryformat "%{name}\t%{distribution}\n" |grep -v "SUSE LINUX 10\.0" (No, I can't fix this, but maybe my hints help the developers to find the problem.)
An ltrace of a failing call would be nice. and its not a glibc problem, it is definitely an rsync problem, likely rsync corrupts memory.
Here's some more stuff as asked: I start with the official rsync, and show that the SuSE one is not installed. rsync --version is provided as you can see they are different based on their capabilities. mc8:~ # rpm--query rsync package rsync is not installed mc8:~ # /usr/local/bin/rsync --version rsync version 2.6.6 protocol version 29 Copyright (C) 1996-2005 by Andrew Tridgell and others <http://rsync.samba.org/> Capabilities: 64-bit files, socketpairs, hard links, symlinks, batchfiles, inplace, IPv6, 64-bit system inums, 64-bit internal inums rsync comes with ABSOLUTELY NO WARRANTY. This is free software, and you are welcome to redistribute it under certain conditions. See the GNU General Public Licence for details. mc8:~ # rm /usr/local/bin/rsync mc8:~ # rsync --version -bash: rsync: command not found mc8:~ # which rsync mc8:~ # rpm -V rsync package rsync is not installed Yes, the old rsync is gone. Now I use yast to install and update the SuSE rsync. mc8:~ # yast mc8:~ # rpm -qa --queryformat "%{name}\t%{distribution}\n" |grep -v "SUSE LINUX 10\.0" gpg-pubkey (none) gpg-pubkey (none) gpg-pubkey (none) gpg-pubkey (none) mc8:~ # rpm -V rsync mc8:~ # rpm --query rsync rsync-2.6.6-2.2 mc8:~ # which rsync /usr/bin/rsync mc8:~ # /usr/local/bin/rsync --version -bash: /usr/local/bin/rsync: No such file or directory Yes, the official one is gone. mc8:~ # backup.spoon.daily building file list ... *** glibc detected *** free(): invalid pointer: 0x00002aaaab11d4a0 *** /usr/local/mis/sbin/backup.spoon.daily: line 2: 18131 Aborted rsync -zav --bwlimit=100 --delete --exclude-from=/usr/local/mis/etc/rsync.exclude --stats / root@10.0.1.20::daily.`hostname -f` mc8:~ # rsync --version rsync version 2.6.6 protocol version 29 Copyright (C) 1996-2005 by Andrew Tridgell and others <http://rsync.samba.org/> Capabilities: 64-bit files, socketpairs, hard links, ACLs, symlinks, batchfiles, inplace, IPv6, 64-bit system inums, 64-bit internal inums, SLP rsync comes with ABSOLUTELY NO WARRANTY. This is free software, and you are welcome to redistribute it under certain conditions. See the GNU General Public Licence for details. This is the output of the SuSE rsync. Here I add ltrace to the script mc8:~ # pico /usr/local/mis/sbin/backup.spoon.daily mc8:~ # cat /usr/local/mis/sbin/backup.spoon.daily #!/bin/bash ltrace -o /tmp/ltrace.txt rsync -zav --bwlimit=100 --delete --exclude-from=/usr/local/mis/etc/rsync.exclude --stats / root@10.0.1.20::daily.`hostname -f` mc8:~ # rq ltrace ltrace-0.3.36-6 mc8:~ # backup.spoon.daily building file list ... *** glibc detected *** free(): invalid pointer: 0x00002aaaab11d4a0 *** still fails and I get a lot of ltrace output. mc8:~ # ls -lah /tmp/ltrace.txt -rw-r--r-- 1 root root 138M 2005-12-31 08:37 /tmp/ltrace.txt mc8:~ # tail /tmp/ltrace.txt strlen("kyoung") = 6 memcpy(0x56b8b6, "2", 1) = 0x56b8b6 memcpy(0x56b8b7, "\005", 1) = 0x56b8b7 memcpy(0x56b8b8, "\006", 1) = 0x56b8b8 memcpy(0x56b8b9, "kyoung", 6) = 0x56b8b9 memcpy(0x56b8bf, "0", 4) = 0x56b8bf memcpy(0x56b8c3, "K\201\347B", 4) = 0x56b8c3 getpwuid(16020, 0x7fffffd1da24, 4, 0x42e7814b, 0xfefeffff00000000 <unfinished ...> --- SIGABRT (Aborted) --- +++ killed by SIGABRT +++ mc8:~ # head /tmp/ltrace.txt __libc_start_main(0x40f8d0, 8, 0x7fffffd25da8, 0x435f20, 0x435fa0 <unfinished ...> signal(10, 0x40f7f0) = NULL signal(12, 0x40f810) = NULL signal(17, 0x40f850) = NULL time(NULL) = 1136036132 geteuid() = 0 memset(0x565be0, '\000', 80) = 0x565be0 umask(00) = 022 malloc(776) = 0x567010 memset(0x567010, '\000', 776) = 0x567010 mc8:~ # mc8:~ # tail /tmp/ltrace.txt strlen("kyoung") = 6 memcpy(0x56b8b6, "2", 1) = 0x56b8b6 memcpy(0x56b8b7, "\005", 1) = 0x56b8b7 memcpy(0x56b8b8, "\006", 1) = 0x56b8b8 memcpy(0x56b8b9, "kyoung", 6) = 0x56b8b9 memcpy(0x56b8bf, "0", 4) = 0x56b8bf memcpy(0x56b8c3, "K\201\347B", 4) = 0x56b8c3 getpwuid(16020, 0x7fffffd1da24, 4, 0x42e7814b, 0xfefeffff00000000 <unfinished ...> --- SIGABRT (Aborted) --- +++ killed by SIGABRT +++ mc8:~ # mc8:~ # scp /tmp/ltrace.txt jp@www.midcoast.com:public_html the ltrace is available at http://www.midcoast.com/~jp/ltrace.txt if needed.
first guess: a duplicate "close" ... (I'm on vacation right now, this might take a few days)
no success yet trying to reproduce ...
could you try to rebuild rsync from our src-rpm with the acl-patches removed ? (these two lines in the specfile:) -patch -p0 < patches/acls.diff -%patch4
I've put the box into service so I won't be able to mess around with different rsyncs now. However, I have a new similar box that will be built up next week with the same configuration and will try as suggested in comment #6.
thanks!
Hi, (pinging again ...) did you see anything on the other box ?
closing until further info, no other report on this problem.