Bug 332917

Created attachment 177672 [details]
Full backtrace

Could please you also run

1) valgrind --log-file=zypper-vg-log /usr/bin/zypper update
2) strace -s 4096 -o zypper-strace-log /usr/bin/zypper update

and attach the logs (compressed if needed)? Thanks!

Jano: I'll look at the curl side to see if the bug is there.

Forgot to set NEEDINFO. See comment #2

The backtrace shows that freedirs() in lib/ftp.c tried to free an invalid pointer conn->data->reqdata.proto.ftp->file. I reviewed the code that sets ->file, but I couldn't find a path where it would assign a non-malloced string permanently (it does so temporarily in ftp_parse_url_path(), case FTPFILE_MULTICWD:, but it duplicates it a few lines below). So I can't tell where the bug comes from atm.

Eskil, the additional logs would help! Please create them ASAP, in case the bug depends on the url that the redirector gives you (which can change at any time).

Sorry, I wasn't able to reproduce it by the time I got your first reply. Hopefully someone who encounters the bug will find this report and the info on how to get the needed logs.

*** Bug 332105 has been marked as a duplicate of this bug. ***

(In reply to comment #5 from Eskil Bylund)
> Sorry, I wasn't able to reproduce it by the time I got your first reply.

I feared that...


Anyway, if you or someone else is able to reproduce it again, could they also try the 7.17.0 version from factory (libcurl4-7.17.0-*.rpm)? Just in case there's a bug that was already fixed upstream.

Created attachment 177999 [details]
Valgrind and strace logs

I couldn't find the package you mentioned but I updated the 10.3 package to 7.17.0 and the crash is still there.

Thanks for the logs!


(In reply to comment #9 from Eskil Bylund)
> I couldn't find the package you mentioned but I updated the 10.3 package to
> 7.17.0 and the crash is still there.

The rpm package was not yet synced out, sorry. Thanks for testing 7.17.0. Just for sure, did you overwrite /usr/lib64/libcurl.so.4* (ie. run ./configure --prefix=/usr --libdir=/usr/lib64)?

Yes. (I updated the 10.3 curl rpm spec file to 7.17.0 and upgraded to the new package.)

Wow...

$ curl -Lv
http://ftp.uninett.no/pub/linux/opensuse/distribution/10.3/repo/debug/content
* About to connect() to ftp.uninett.no port 80 (#0)
*   Trying 158.36.2.10... connected
* Connected to ftp.uninett.no (158.36.2.10) port 80 (#0)
> GET /pub/linux/opensuse/distribution/10.3/repo/debug/content HTTP/1.1
> User-Agent: curl/7.17.0 (x86_64-suse-linux-gnu) libcurl/7.17.0 OpenSSL/0.9.8e zlib/1.2.3 libidn/1.0
> Host: ftp.uninett.no
> Accept: */*
> 
< HTTP/1.1 302 Found
< Date: Fri, 12 Oct 2007 11:49:37 GMT
< Server: Apache/1.3.37 (Unix) PHP/4.4.4 mod_perl/1.29
< Location:
ftp://ftp.uninett.no/pub/linux/opensuse/distribution/10.3/repo/debug/content
< Transfer-Encoding: chunked
< Content-Type: text/html; charset=iso-8859-1
< 
* Ignoring the response-body
* Connection #0 to host ftp.uninett.no left intact
* Issue another request to this URL:
'ftp://ftp.uninett.no/pub/linux/opensuse/distribution/10.3/repo/debug/content'
* About to connect() to ftp.uninett.no port 21 (#1)
*   Trying 158.36.2.10... connected
* Connected to ftp.uninett.no (158.36.2.10) port 21 (#1)
*** glibc detected *** curl: free(): invalid pointer: 0x00002b01a33115f2 ***
...

The strace helped. I also have a suspect why it happens. Stay tuned.

Peter, does it make sense to disable ftp hosts in the redirector as a temporary woraround? I mean I hope to fix it shortly, but it'll take some time until the update is available...

Darix? Is it possible to stop redirecting to mirrors that do http -> ftp redirects?

OK, the ftp.uninett.no mirror should be disabled now, thanks Darix. If you hit the bug again, please run zypper under strace, look for any 'Location: ftp://' string in the strace output and report the mirror here.

As for the bug itself: I've a workaround for it, discussing a proper fix with upstream [*]. I'm building testing packages in the buildservice, they'll appear here once built: http://download.opensuse.org/repositories/home:/michal-m/openSUSE_10.3 (only libcurl4.rpm was changed).


[*] http://curl.haxx.se/mail/lib-2007-10/index.html#122

There is one mirror (ftp.is.co.za) which is ftp-only (i.e., we
immediately redirect to ftp://ftp.is.co.za/mirror/opensuse/opensuse/
ourselves. (They have already set up an Apache, I am in contact with
them about that.) Should I disable redirection to them as well? 

Did someone contact uninett.no? If not, I'll talk to them.

(In reply to comment #16 from Peter Poeml)
> There is one mirror (ftp.is.co.za) which is ftp-only (i.e., we
> immediately redirect to ftp://ftp.is.co.za/mirror/opensuse/opensuse/
> ourselves. (They have already set up an Apache, I am in contact with
> them about that.) Should I disable redirection to them as well?

Yes, please do. It's the http -> ftp redirect that libcurl can't handle on 10.3


> Did someone contact uninett.no? If not, I'll talk to them.

Not me. But their setup is perfectly ok, just our tool can't handle it :-/


(In reply to comment #15 from Michal Marek)
> As for the bug itself: I've a workaround for it, discussing a proper fix with
> upstream [*]. I'm building testing packages in the buildservice

Sorry, that didn't work at all. Unfortunately the patch from upstream didn't
help either. Stay tuned...

Okay, ftp.is.co.za is disabled for now.

I've a patch against CVS HEAD that fixes this, I'm going to backport it to the version used in 10.3.

I finally have testing packages in http://download.opensuse.org/repositories/home:/michal-m/openSUSE_10.3/. I tried adding repos from ftp://ftp.uninett.no/pub/linux/opensuse/distribution/10.3/repo/ and installing packages and all went ok.

Anja, can I get a SWAMP-id for a 10.3 update?

I submitted a fixed curl package to autobuild for 10.3 update. However, we probably shouldn't re-enable the disabled mirrors, because that would break 10.3 GM installations (network installations and adding online repos during install).

Created attachment 180103 [details]
checking mirrors for FTP redirects

As the attached transcript shows, there is no mirror which redirects to
FTP servers inside the 10.3 repo. So we should be fine for now.

released

*** Bug 333390 has been marked as a duplicate of this bug. ***

*** Bug 333639 has been marked as a duplicate of this bug. ***

reset-reqproto.patch references data after free, which causes git-fetch to crash when using http transport.

Created attachment 210487 [details]
Testcase

$ make -s curl LDLIBS=-lcurl
$ valgrind ./curl
==17049== Memcheck, a memory error detector.
==17049== Copyright (C) 2002-2007, and GNU GPL'd, by Julian Seward et al.
==17049== Using LibVEX rev 1732, a library for dynamic binary translation.
==17049== Copyright (C) 2004-2007, and GNU GPL'd, by OpenWorks LLP.
==17049== Using valgrind-3.2.3, a dynamic binary instrumentation framework.
==17049== Copyright (C) 2000-2007, and GNU GPL'd, by Julian Seward et al.
==17049== For more details, rerun with: -v
==17049== 
4d3a72da3f21761bf4f2866a6a3400d4ecf67365        refs/heads/master
==17049== Invalid read of size 4
==17049==    at 0xFF6B360: conn_free (url.c:1796)
==17049==    by 0xFF6E040: Curl_rm_connc (url.c:487)
==17049==    by 0xFF8143C: curl_multi_cleanup (multi.c:1557)
==17049==    by 0x10001C6C: main (curl.c:46)
==17049==  Address 0x450C57C is 332 bytes inside a block of size 34,176 free'd
==17049==    at 0xFFB8A9C: free (vg_replace_malloc.c:233)
==17049==    by 0xFF713B8: Curl_close (url.c:384)
==17049==    by 0xFF7CE48: curl_easy_cleanup (easy.c:507)
==17049==    by 0x10001C64: main (curl.c:45)
==17049== 
==17049== Invalid write of size 4
==17049==    at 0xFF6B454: conn_free (url.c:1797)
==17049==    by 0xFF6E040: Curl_rm_connc (url.c:487)
==17049==    by 0xFF8143C: curl_multi_cleanup (multi.c:1557)
==17049==    by 0x10001C6C: main (curl.c:46)
==17049==  Address 0x450C57C is 332 bytes inside a block of size 34,176 free'd
==17049==    at 0xFFB8A9C: free (vg_replace_malloc.c:233)
==17049==    by 0xFF713B8: Curl_close (url.c:384)
==17049==    by 0xFF7CE48: curl_easy_cleanup (easy.c:507)
==17049==    by 0x10001C64: main (curl.c:45)
==17049== 
==17049== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 5 from 2)
==17049== malloc/free: in use at exit: 756 bytes in 11 blocks.
==17049== malloc/free: 3,040 allocs, 3,029 frees, 96,621 bytes allocated.
==17049== For counts of detected errors, rerun with: -v
==17049== searching for pointers to 11 not-freed blocks.
==17049== checked 157,464 bytes.
==17049== 
==17049== LEAK SUMMARY:
==17049==    definitely lost: 0 bytes in 0 blocks.
==17049==      possibly lost: 0 bytes in 0 blocks.
==17049==    still reachable: 756 bytes in 11 blocks.
==17049==         suppressed: 0 bytes in 0 blocks.
==17049== Rerun with --leak-check=full to see details of leaked memory.

Does http://cool.haxx.se/cvs.cgi/curl/lib/url.c.diff?r1=1.669&r2=1.670 fix the git-fetch crash?

SWAMPID: 17481
for releasing an update.

Submitted a fixed package.

update released