Bug 136794

Please, attach y2logs (see http://www.opensuse.org/Bug_Reporting_FAQ#YaST).

Created attachment 59925 [details]
YaST2 logs

 MediaCurl.cc(getFileCopy):432 curl error: 7: couldn't connect to host'

The mirror was not reachable then.

> The mirror was not reachable then.

For YaST2 to even start, an initrd file must be loaded first, which comes from the ftp mirror. The fact that YaST2 started up succesfully proves that the ftp mirror was up and running and could be reached at that moment.

Also, I tried this on two different days, using two different mirrors. The first day I double-checked the datapaths on the ftp mirrors right before I started the installation. Obviously both ftp mirrors could be reached normally at that time. It would seem extremely unlikely that on each attempt to install (at least 4 times that I remember), access would be normal right to the moment the initrd file was loaded, and then all of a sudden the ftp mirror would become unreachable....

IMO this either points to a problem in MediaCurl (or any of its subsidiaries) or to an authentication problem causing the connection to be refused by the server. Either way this is beyond my control and needs to be fixed by you. I would therefore like to ask you to reopen this bug.

This problem is clearly specific to curl, and has nothing to do with reachability of the ftp server, as is shown by this little test trying to obtain one of the files as listed in the y2log

> curl ftp://134.60.1.5/%2fpub/mirrors/opensuse/distribution/SL-10.0-OSS/inst-source/media.1/products
curl: (7) couldn't connect to host
> wget ftp://134.60.1.5/%2fpub/mirrors/opensuse/distribution/SL-10.0-OSS/inst-source/media.1/products
--18:41:16--  ftp://134.60.1.5/%2fpub/mirrors/opensuse/distribution/SL-10.0-OSS/inst-source/media.1/products
           => `products'
Connecting to 134.60.1.5:21... connected.
Logging in as anonymous ... Logged in!
==> SYST ... done.    ==> PWD ... done.
==> TYPE I ... done.  ==> CWD /pub/mirrors/opensuse/distribution/SL-10.0-OSS/inst-source/media.1 ... done.
==> PASV ... done.    ==> RETR products ... done.
Length: 26 (unauthoritative)

100%[========================================================================================================================>] 26            --.--K/s

18:41:16 (1.50 KB/s) - `products' saved [26]

> curl ftp://134.60.1.5/%2fpub/mirrors/opensuse/distribution/SL-10.0-OSS/inst-source/media.1/products
curl: (7) couldn't connect to host

curl cannot connect to the host, while wget is perfectly capable of obtaining the exact same URL. Needless to say that these tests were performed only seconds apart...

It works for me here both with wget and curl. Please try again with curl --verbose.

> curl --verbose ftp://134.60.1.5/%2fpub/mirrors/opensuse/distribution/SL-10.0-OSS/inst-source/media.1/products
* About to connect() to 134.60.1.5 port 21
*   Trying 134.60.1.5... * connected
* Connected to 134.60.1.5 (134.60.1.5) port 21
< 220 (vsFTPd 2.0.3)
> USER anonymous
< 331 Please specify the password.
> PASS curl_by_daniel@haxx.se
< 230 Login successful.
* We have successfully logged in
> PWD
< 257 "/"
* Entry path is '/'
> CWD /pub
< 250-***************************************************************************
< 250-
< 250-This directory tree is under serious reconstruction!
< 250-
< 250-If you cant find what you are looking for it may well be moved to
< 250-/pub/misc until its final destination is considered.
< 250-
< 250-***************************************************************************
< 250 Directory successfully changed.
> CWD mirrors
< 250 Directory successfully changed.
> CWD opensuse
< 250 Directory successfully changed.
> CWD distribution
< 250 Directory successfully changed.
> CWD SL-10.0-OSS
< 250 Directory successfully changed.
> CWD inst-source
< 250 Directory successfully changed.
> CWD media.1
< 250 Directory successfully changed.
> EPSV
< 229 Entering Extended Passive Mode (|||25769|)
*   Trying 134.60.1.5... * Connection refused
* couldn't connect to host
* Remembering we are in dir /pub/mirrors/opensuse/distribution/SL-10.0-OSS/inst-source/media.1/products
* Connection #0 to host 134.60.1.5 left intact
curl: (7) couldn't connect to host
* Closing connection #0

I checked it on various machines internal as well as external (SL10.0, SL9.3, SL 9.1, SLES9 and even debian sarge) ... and (I'm sorry) it always worked for me.

What system did you do the tests of comment 5 on?
btw. Did you already perform a media check of your miniiso?

The tests were peformed on a fully patched SuSE 9.3 professional system (IA32). The media check of the miniiso was one of the first things I did. It is OK.

I ran wget in verbose mode and noticed that it used PASV mode rather than EPSV, so I tried the following:

> curl --disable-epsv --verbose ftp://134.60.1.5/%2fpub/mirrors/opensuse/distribution/SL-10.0-OSS/inst-source/media.1/products
* About to connect() to 134.60.1.5 port 21
*   Trying 134.60.1.5... * connected
* Connected to 134.60.1.5 (134.60.1.5) port 21
< 220 (vsFTPd 2.0.3)
> USER anonymous
< 331 Please specify the password.
> PASS curl_by_daniel@haxx.se
< 230 Login successful.
* We have successfully logged in
> PWD
< 257 "/"
* Entry path is '/'
> CWD /pub
< 250-***************************************************************************
< 250-
< 250-This directory tree is under serious reconstruction!
< 250-
< 250-If you cant find what you are looking for it may well be moved to
< 250-/pub/misc until its final destination is considered.
< 250-
< 250-***************************************************************************
< 250 Directory successfully changed.
> CWD mirrors
< 250 Directory successfully changed.
> CWD opensuse
< 250 Directory successfully changed.
> CWD distribution
< 250 Directory successfully changed.
> CWD SL-10.0-OSS
< 250 Directory successfully changed.
> CWD inst-source
< 250 Directory successfully changed.
> CWD media.1
< 250 Directory successfully changed.
> PASV
< 227 Entering Passive Mode (134,60,1,5,158,192)
*   Trying 134.60.1.5... * connected
* Connecting to 134.60.1.5 (134.60.1.5) port 40640
* Connected the data stream with PASV!
> TYPE I
< 200 Switching to Binary mode.
> SIZE products
< 213 26
> RETR products
< 150 Opening BINARY mode data connection for products (26 bytes).
* Getting file with size: 26
/ SuSE-Linux-FTP-OSS 10.0
* Remembering we are in dir /pub/mirrors/opensuse/distribution/SL-10.0-OSS/inst-source/media.1/
< 226 File send OK.
* Connection #0 to host 134.60.1.5 left intact
> QUIT
< 221 Goodbye.
* Closing connection #0

So it seems that EPSV does not work, while PASV does. According to the man pages: "Curl will normally always first attempt to use  EPSV  before  PASV, ...". It seems that in practise however curl gives up after trying EPSV and never gets to the stage where it would try PASV mode.

Is this a bug in curl? Should it have tried PASV mode after the failure of EPSV mode?

I guess there is no way to tell the miniiso to use PASV mode rather than EPSV mode?

@mmarek: is this a bug in curl?

Maybe. May I ask for strace output?

strace -o curl.strace curl --verbose \
ftp://134.60.1.5/%2fpub/mirrors/opensuse/distribution/SL-10.0-OSS/inst-source/media.1/products

It seems like send()ing "EPSV\r\n" to the server fails... (maybe some firewall in the way which tries to filter ftp?)

Created attachment 63643 [details]
strace output

It is certainly possible that our firewall gets in the way of establishing an EPSV connection...

Could you please repeat your tests with 

ftp://ftp.suse.com/pub/people/mmarek/bug136794/curl*.rpm

? Thanks. The packages are built for 9.2-i586. It should fallback to PASV
even if the server has replied to EPSV.

This time it worked OK!

> curl --verbose ftp://134.60.1.5/%2fpub/mirrors/opensuse/distribution/SL-10.0-OSS/inst-source/media.1/products
* About to connect() to 134.60.1.5 port 21
*   Trying 134.60.1.5... connected
* Connected to 134.60.1.5 (134.60.1.5) port 21
< 220 (vsFTPd 2.0.3)
> USER anonymous
< 331 Please specify the password.
> PASS curl_by_daniel@haxx.se
< 230 Login successful.
> PWD
< 257 "/"
* Entry path is '/'
> CWD /pub
< 250-***************************************************************************
< 250-
< 250-This directory tree is under serious reconstruction!
< 250-
< 250-If you cant find what you are looking for it may well be moved to
< 250-/pub/misc until its final destination is considered.
< 250-
< 250-***************************************************************************
< 250 Directory successfully changed.
> CWD mirrors
< 250 Directory successfully changed.
> CWD opensuse
< 250 Directory successfully changed.
> CWD distribution
< 250 Directory successfully changed.
> CWD SL-10.0-OSS
< 250 Directory successfully changed.
> CWD inst-source
< 250 Directory successfully changed.
> CWD media.1
< 250 Directory successfully changed.
> EPSV
* Connect data stream passively
< 229 Entering Extended Passive Mode (|||20352|)
*   Trying 134.60.1.5... Connection refused
* couldn't connect to host
* got positive EPSV response, but can't connect
* disabling EPSV usage
> PASV
< 227 Entering Passive Mode (134,60,1,5,23,237)
*   Trying 134.60.1.5... connected
* Connecting to 134.60.1.5 (134.60.1.5) port 6125
> TYPE I
< 200 Switching to Binary mode.
> SIZE products
< 213 26
> RETR products
< 150 Opening BINARY mode data connection for products (26 bytes).
* Getting file with size: 26
/ SuSE-Linux-FTP-OSS 10.0
* Remembering we are in dir /pub/mirrors/opensuse/distribution/SL-10.0-OSS/inst-source/media.1/
< 226 File send OK.
* Connection #0 to host 134.60.1.5 left intact
> QUIT
< 221 Goodbye.
* Closing connection #0

Created attachment 64572 [details]
strace output from curl 7.15.1

Thanks!  Let's add this updated curl package to our distro.

I wonder why not one person who worked on this issue ever bothered to assume ownership of this bug.

The patched package is finally in opensuse factory. If you are brave enough,
you can try whether the installation works now. I'm marking this as fixed.