Bug 555066

Summary: aria2 improperly ignores no_proxy environment *and* /etc/sysconfig/proxy settings
Product: [openSUSE] openSUSE 11.2 Reporter: Jon Nelson <jnelson-suse>
Component: libzyppAssignee: E-mail List <zypp-maintainers>
Status: RESOLVED FIXED QA Contact: E-mail List <qa-bugs>
Severity: Normal    
Priority: P3 - Medium CC: binner, erico.mendonca, forgotten_xI2C5NvggO, ma, pascal.bleser, tatsuhiro.t
Version: Final   
Target Milestone: ---   
Hardware: x86-64   
OS: Other   
Whiteboard:
Found By: --- Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---

Description Jon Nelson 2009-11-12 21:43:11 UTC
User-Agent:       Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.15) Gecko/2009102100 SUSE/3.0.15-0.1.2 Firefox/3.0.15

aria2c's downloader appears to be ignoring the no_proxy environment variable. The variable usually but not always comes from /etc/sysconfig/proxy.

Strictly, aria2c should *not* be invoked with --http-proxy=XXX as it consults the environment for that, which usually BUT NOT ALWAYS comes from /etc/sysconfig/proxy.  Even when the proxy was turned *off* in /etc/sysconfig/proxy, the environment variable http_proxy was used by aria2c, but the no_proxy environment variable was ignored (or appeared to be).

This is an issue for many corporate environments which mandate the use of a proxy for external resources but that same proxy will not work for internal resources.



Reproducible: Always

Steps to Reproduce:
1.
2.
3.
Comment 1 Michael Andres 2009-11-18 14:10:36 UTC
Yes, in zypp /etc/sysconfig/proxy overrules the environment. Might be this is done to immediately honor any changes done to the proxy settings via YaST. We have to check this.



> /etc/sysconfig/proxy, the environment variable http_proxy was used by aria2c,
> but the no_proxy environment variable was ignored (or appeared to be).

Can't verify this. aria2 (1.6.3) respects the no_proxy environment variable if it is set (and not overruled by commandline options).
Comment 2 Jon Nelson 2009-11-24 16:04:47 UTC
This is the sequence I went through. Perhaps it is useful.

I noticed that my firewall proxy was getting requests for semi-internal resource (a mounted openSUSE 11.2 DVD .iso, used to avoid network traffic).  The resource being requested was http://192.168.2.1/~jnelson/....

However, 192.168.2.0/24 is in no_proxy:


jnelson@turnip:~> env | grep prox
http_proxy=http://192.168.1.1:3128/
ftp_proxy=http://192.168.1.1:3128/
https_proxy=http://192.168.1.1:3128/
no_proxy=localhost,192.168.2.0/24
jnelson@turnip:~>

Additionally, when you zypper is downloading stuff, you can in another terminal 'ps waux | grep aria2'

and see the '--http-proxy=...' commandline argument being used.

Thus, there are 2 issues:

1. the --http-proxy commandline argument is unnecessary, if zypper is getting the values from /etc/sysconfig/proxy - if this is true, then the environment would already have the correct values.  It is reasonable to use this commandline argument if zypper gets full proxy config in /etc/zypp/zypp{,er}.conf 

2. aria2c *appears* to be ignoring or mis-handling the no_proxy environment variable. The network 192.168.2.0/24 should not have been accessed through the proxy, but it was.


Easy to test. Replicate thusly (obviously, you'll need to change for your environment):

env http_proxy=http://192.168.1.1:3128/ no_proxy=localhost,192.168.2.0/24 aria2c --log-level=debug --file-allocation=none http://192.168.2.1/~jnelson/isos/openSUSE-11.2-DVD-x86_64.iso

and while that is running:

netstat -planetu | grep aria2c

As you can see, it's got (5 by default) connections to port 3128 on 192.168.1.1 when it should have none.
Comment 3 Jon Nelson 2009-11-24 16:07:05 UTC
One thing to note:

the latest version of aria2 available is *1.5.2*

worklaptop:~ # zypper search -s aria2
Loading repository data...
Reading installed packages...

S | Name              | Type    | Version     | Arch   | Repository
--+-------------------+---------+-------------+--------+--------------------
i | aria2             | package | 1.5.2-2.3.1 | x86_64 | openSUSE 11.2 (DVD)
i | aria2             | package | 1.5.2-2.3.1 | x86_64 | openSUSE-11.2-Oss
v | aria2             | package | 1.5.2-2.3.1 | i586   | openSUSE-11.2-Oss
  | aria2-debuginfo   | package | 1.5.2-2.3.1 | x86_64 | openSUSE-11.2-Debug
  | aria2-debuginfo   | package | 1.5.2-2.3.1 | i586   | openSUSE-11.2-Debug
  | aria2-debugsource | package | 1.5.2-2.3.1 | x86_64 | openSUSE-11.2-Debug
  | aria2-debugsource | package | 1.5.2-2.3.1 | i586   | openSUSE-11.2-Debug
worklaptop:~ #
Comment 4 Michael Andres 2009-11-26 17:28:43 UTC
Frankly, I never used a netmask like 192.168.2.0/24 in no_proxy. Maybe that's the reason.
Comment 5 Jon Nelson 2009-11-28 16:53:49 UTC
A few notes:

- the source for aria2 appears to have fairly weak proxy parsing. it *only* supports FQDN or FQDN-like matching. It doesn't match IPs or CIDRs, and it doesn't match the (optional) port specification either.

The most common standard is the mozilla one:

https://developer.mozilla.org/en/No_Proxy_For_configuration

The most likely expectations, IMO, are these:

1. expand names to IP addresses (as appropriate)
2. compare each IP address to each item in the no_proxy list, if the item is a CIDR perform the appropriate comparison. 

First match always wins.
Comment 6 Pascal Bleser 2010-02-09 20:08:12 UTC
Hmh, well, changing aria2's no_proxy parsing isn't that trivial I'm afraid.

But 1.8.2 should actually honor no_proxy:

src/option_processing.cc
165:    overrideWithEnv(op, oparser, PREF_NO_PROXY, "no_proxy");

Could you test again with the aria2-1.8.2 package from the network:utilities repository ?
Comment 7 Tatsuhiro Tsujikawa 2010-03-22 10:05:19 UTC
In the development code, we made aria2 accept network address with CIDR in --no-proxy option(and no_proxy env variable). So the next release 1.9.1 has this feature.
Comment 8 Jon Nelson 2010-07-25 18:27:15 UTC
Looks like 1.9.1 comes with openSUSE 11.3. 

This bug can probably be closed.
Comment 9 Michael Andres 2010-07-26 09:21:46 UTC
.