|
Bugzilla – Full Text Bug Listing |
| Summary: | rpm fails to update packages when a certain total size is exceeded | ||
|---|---|---|---|
| Product: | [openSUSE] SUSE Linux 10.1 | Reporter: | Marvin FourtyTwo <marvin24> |
| Component: | Basesystem | Assignee: | E-mail List <kernel-maintainers> |
| Status: | RESOLVED FIXED | QA Contact: | E-mail List <qa-bugs> |
| Severity: | Critical | ||
| Priority: | P5 - None | CC: | suse-beta |
| Version: | Beta 5 | ||
| Target Milestone: | --- | ||
| Hardware: | 32bit | ||
| OS: | SuSE Linux 10.1 | ||
| Whiteboard: | |||
| Found By: | Other | Services Priority: | |
| Business Priority: | Blocker: | --- | |
| Marketing QA Status: | --- | IT Deployment: | --- |
| Attachments: |
last 100 lines of rpm -Fvvh kde* g* p*
last 1000 lines of ltrace -Sf rpm -Fvvh ... |
||
|
Description
Marvin FourtyTwo
2006-02-22 13:47:01 UTC
Raising severity. stops? stops with an error code? please run rpm with the -vv option so that it prints some debug messages. It is *not* easy to reproduce, installation works fine here (I just did an 11 GByte update). (In reply to comment #2) > stops? stops with an error code? please run rpm with the -vv option so that > it prints some debug messages. > It is *not* easy to reproduce, installation works fine here (I just did an 11 > GByte update). > wow - where did you find all these packages? I have attached the final lines of rpm -Fvvh kde* g* p*. Together ~230MB of packages from beta4 to current-factory (from this morning I think). Btw. this bug happens since beta1, the first version of opensuse I tested. Also attached is the last output of ltrace -Sf of the same command. The full logs can be found here: http://www.uni-giessen.de/~marc/152779 (several MB!). The first package which is going to be updated is pango. From the debug output, it seems, that the pre-install script fails (maybe because /etc/pango and /usr/lib/pango) don't exist. Then, rpm just quits (returning 0?) I'm wondering, if I am the only one, who gets this bug. Maybe I have a strange system configuration... could this be a reason? Created attachment 69875 [details]
last 100 lines of rpm -Fvvh kde* g* p*
Created attachment 69876 [details]
last 1000 lines of ltrace -Sf rpm -Fvvh ...
> bug happens since beta1, the first version of opensuse I tested. > Also attached is the last output of ltrace -Sf of the same command. > The full logs can be found here: http://www.uni-giessen.de/~marc/152779 > (several MB!). errr - the url of the full logs is http://wwwiap.physik.uni-giessen.de/~marc/152779 Actually I don't understand the ltrace. It only runs the preinstall script. A very suspicious line is SYS_clone(0x01200011, 0, 0, 0, 0xb7b926f8) = -12 which looks to me like the clone call (aka fork()) didn't work for some reason. Hmm, ENOMEM is 12, this could be ltrace's way of telling us that fork returned with ENOMEM. Maybe rpm cannot deal with that correctly. Do you have any limits enabled? I.e., what's the output of 'ulimit -a'? Or no swap configured? (In reply to comment #7) > Do you have any limits enabled? I.e., what's the output of 'ulimit -a'? > Or no swap configured? as far as I know, I didn't changed anything, but here it is: core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited file size (blocks, -f) unlimited pending signals (-i) 3583 max locked memory (kbytes, -l) 32 max memory size (kbytes, -m) unlimited open files (-n) 1024 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 3583 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited maybe I need to say, that I run 2.6.16-rc4-mm1, but this should be unrelated, because it also happend to other kernels too ... I have 450MB real and 512MB swap. when running rpm -Fvvh kde* ..., it takes ~250MB of ram and system starts to swap a little. I have only 90MB left on the root partion. I'm pretty sure that the problem is that the fork() call fails with ENOMEM. I checked the rpm source code and it can't deal with a failing fork and just assumes that it is in the child process, which is exactly the behavior you are experiencing. Could you run 'free' a couple of times when rpm is running to check the memory usage? Is there any message in /var/log/messages? (In reply to comment #9) > I'm pretty sure that the problem is that the fork() call fails with ENOMEM. I > checked the rpm source code and it can't deal with a failing fork and just > assumes that it is in the child process, which is exactly the behavior you are > experiencing. > Could you run 'free' a couple of times when rpm is running to check the memory > usage? Is there any message in /var/log/messages? no - nothing here is the output of vmstat -n 1 procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu---- r b swpd free buff cache si so bi bo in cs us sy id wa 0 0 94648 14708 75268 134476 2 3 56 23 18 30 8 1 90 1 0 0 94648 14708 75268 134476 0 0 0 0 335 204 0 0 100 0 0 1 94648 8668 75340 137660 0 0 3244 0 405 477 26 8 36 30 0 1 94648 5880 75368 139568 0 0 4240 0 448 382 17 6 0 77 0 1 94648 5832 75284 140356 0 0 4636 0 451 418 24 11 0 66 0 1 94648 6152 75148 140760 0 0 4108 0 431 343 33 8 0 59 0 1 94648 5484 75024 141852 0 0 4680 0 433 373 26 9 0 65 0 1 94648 5756 74936 141772 0 0 3800 0 392 297 51 8 0 41 1 1 94648 5276 74892 140924 0 0 3708 0 621 533 77 12 0 11 1 1 94648 5500 74864 138572 0 0 3508 0 429 1764 83 17 0 0 1 1 94648 5652 74864 137700 0 0 2840 0 416 334 90 10 0 0 1 1 94648 6096 74860 137648 0 0 2632 0 401 315 93 7 0 0 2 1 94648 18100 74888 138808 0 0 1908 48 404 393 94 6 0 0 0 1 94648 14768 74888 142072 0 0 3264 0 428 456 47 5 0 48 1 1 94648 9380 74892 145960 0 0 3892 0 420 335 37 5 0 58 1 0 94648 7780 73104 149964 0 0 5644 0 496 397 31 18 0 51 0 1 94648 5732 72540 153508 0 0 3620 0 436 376 41 16 0 43 1 0 94648 6288 70844 155496 0 0 2016 260 443 384 37 14 0 50 0 1 94648 5700 70352 157920 0 0 3104 0 444 364 44 9 0 48 4 0 94648 6036 69632 159016 0 0 1528 0 388 271 71 8 0 21 1 0 94648 5896 69632 159256 0 0 240 0 339 203 97 2 0 1 1 0 94648 7040 69360 158420 0 0 1616 0 390 274 75 8 0 17 2 0 94648 5960 69372 159508 0 0 1088 20 367 258 86 3 0 11 1 0 94648 5796 69044 160204 0 0 1008 0 374 255 78 3 0 19 2 0 94648 5752 69044 160300 0 0 96 0 333 198 95 3 0 2 0 1 94648 5152 69044 160860 0 0 560 0 350 226 85 5 0 10 1 0 94648 7848 67496 160652 0 0 932 0 365 253 76 13 0 11 2 0 94648 5428 67496 162408 0 0 1756 0 403 323 62 15 0 23 1 0 94648 5204 66620 165160 0 0 3156 0 417 306 74 7 0 19 1 0 94648 5816 66288 165324 0 0 600 0 360 242 78 11 0 11 1 0 94648 10584 66292 165428 0 0 132 0 334 198 91 9 0 0 0 1 94648 9096 66640 165464 0 0 384 0 420 366 52 6 0 42 0 1 94648 7608 67220 165464 0 0 548 56 467 683 0 3 0 97 3 1 94648 6864 67748 165464 0 0 528 0 466 468 1 5 0 94 1 1 94648 6120 68368 165464 0 0 620 0 486 490 2 3 0 95 0 1 94648 5132 69040 165464 0 0 672 0 498 536 1 4 0 95 0 1 94648 5560 69092 165012 0 0 624 0 484 490 2 5 0 93 1 1 94648 5864 69356 164792 0 0 684 0 499 532 1 5 0 94 1 1 94648 5248 69892 164792 0 0 536 0 463 450 1 2 0 97 1 1 94648 5820 69920 164792 0 0 540 0 463 463 1 4 0 95 0 1 94648 5996 69916 164476 0 0 576 0 474 615 6 5 0 89 0 1 94648 5376 70416 164476 0 0 500 0 455 430 15 3 0 82 0 1 94648 5732 68892 165944 0 0 1860 0 464 583 17 8 0 75 0 1 94648 5944 67992 167524 0 0 2072 0 470 461 7 10 0 83 0 1 94648 10260 67836 163712 0 0 1840 0 454 428 20 6 0 74 1 0 94648 7968 67836 165668 0 0 1956 0 420 333 50 13 0 37 0 1 94648 6832 67836 166900 0 0 1232 0 412 339 33 15 0 52 2 0 94648 6292 67852 167432 0 0 528 0 373 289 64 15 0 21 1 0 94648 5932 67944 167792 0 0 284 0 369 264 61 26 0 13 1 0 94648 5932 67944 167792 0 0 0 0 328 196 72 28 0 0 1 0 94648 7648 65960 167104 0 0 16 0 333 218 69 30 0 1 1 0 94648 7648 65976 167260 0 0 144 0 336 237 71 27 0 2 1 0 94648 7400 65976 167260 0 0 0 0 330 215 75 25 0 0 1 0 94648 7464 65984 167292 0 0 20 0 339 225 73 25 0 2 2 0 94648 7464 65984 167292 0 0 0 0 330 213 69 31 0 0 1 0 94648 7472 65984 167292 0 0 0 0 328 217 68 32 0 0 2 0 94648 7352 66000 167436 0 0 124 0 346 237 60 30 0 10 2 0 94648 5704 66012 168476 0 0 1052 0 411 309 76 19 0 5 1 0 94648 5704 66012 168484 0 0 8 0 331 211 73 26 0 1 1 0 94648 5644 66016 168508 0 0 20 0 339 212 71 27 0 2 1 0 94648 6736 65416 168260 0 0 164 0 353 220 70 24 0 6 1 0 94648 6740 65416 168260 0 0 0 0 328 198 70 30 0 0 1 0 94648 6740 65460 168260 0 0 0 88 343 204 78 22 0 0 1 0 94648 6740 65460 168260 0 0 0 0 328 207 70 30 0 0 1 1 94648 6740 65480 168428 0 0 172 0 343 219 76 19 0 5 0 1 94648 7572 64248 167808 0 0 60 0 343 227 73 22 0 5 2 0 94648 7572 64252 167824 0 0 16 0 332 195 64 34 0 2 0 1 94648 7512 64288 168044 0 0 184 0 354 255 70 22 0 8 1 0 94648 8008 64292 168048 0 0 4 0 341 198 68 32 0 0 4 0 94648 8008 64292 168048 0 0 0 0 329 197 74 26 0 0 1 0 94648 8008 64292 168048 0 0 0 0 330 187 72 28 0 0 2 0 94648 5264 63024 167524 0 0 0 0 328 198 72 28 0 0 1 0 94648 5604 56716 166292 0 0 0 20 332 219 66 34 0 0 1 0 94648 6180 55128 164416 0 0 0 0 332 199 70 30 0 0 1 0 94648 5600 55060 159620 0 0 0 0 329 207 67 33 0 0 1 0 94648 5640 54996 153924 0 0 0 0 328 218 67 33 0 0 2 0 94648 5816 54988 149452 0 0 0 0 332 196 70 30 0 0 1 0 94648 5484 54704 144680 0 0 0 0 331 211 68 32 0 0 1 0 94648 6132 53752 140536 0 0 0 0 331 201 69 31 0 0 1 0 94648 5884 49112 140336 0 0 0 0 328 211 70 30 0 0 1 0 94648 5472 45076 137332 0 0 0 0 328 201 68 32 0 0 1 0 94648 5212 42888 133120 0 0 0 0 330 205 69 31 0 0 1 1 94648 5332 41612 122700 0 0 196 16 341 245 55 39 0 6 2 0 94648 5168 41488 113820 0 0 0 0 335 233 61 39 0 0 3 0 94648 6224 41488 102884 0 0 708 0 358 260 55 38 0 7 1 0 94648 5552 38204 68740 0 0 16 12 343 317 56 44 0 0 2 1 94648 4480 22584 66360 0 0 588 0 371 315 56 25 0 19 1 1 94648 5072 13372 66532 0 0 212 0 333 246 64 33 0 3 2 1 94648 4820 5128 65292 0 0 996 0 365 259 60 33 0 7 1 1 94648 4752 3068 61644 0 0 104 0 335 236 32 33 0 35 1 1 94648 4660 3008 54072 0 0 184 0 342 224 56 33 0 11 1 1 94648 4692 3016 47600 0 0 316 0 353 255 60 28 0 12 1 1 94648 5136 972 48896 0 0 5220 0 474 383 30 16 0 54 0 3 94648 6196 800 44712 0 0 1500 0 370 290 31 22 0 47 1 1 94648 4472 764 38088 0 0 464 0 369 283 45 34 0 21 2 1 94896 5184 764 34660 0 1736 124 1736 566 260 39 22 0 38 2 2 95024 4548 768 33384 0 1072 2904 1072 566 335 27 21 0 52 0 2 95028 4808 792 31732 0 368 344 368 422 278 28 16 0 56 0 1 95976 5924 760 29576 0 1436 2416 1440 515 333 26 18 0 56 2 1 97488 5160 760 29920 0 1628 688 1628 414 324 22 15 0 63 2 1 102380 5280 768 30604 0 4972 708 4972 410 340 32 21 0 46 2 0 106344 5732 812 31268 0 4752 776 4752 543 340 43 28 0 29 1 1 110088 7276 860 31708 96 4132 700 4132 485 361 30 19 0 51 1 0 95440 312376 1004 32780 0 0 1176 0 430 301 0 12 43 46 0 0 95440 312376 1004 32780 0 0 0 0 330 187 0 0 100 0 0 0 95440 312384 1004 32780 0 0 0 0 328 196 0 0 100 0 1 0 95440 312384 1004 32780 0 0 0 0 328 180 0 0 100 0 0 0 95440 312404 1036 32780 0 0 0 56 337 227 0 0 100 0 2 0 95440 312404 1036 32780 0 0 0 0 328 179 0 0 100 0 0 0 95440 312404 1048 32780 148 0 160 0 353 371 4 0 86 10 Hmm, only 100M swap used, are you sure you have 512MB swap configured? Does the installation work if you do echo 1 > /proc/sys/vm/overcommit_memory (In reply to comment #11) > Hmm, only 100M swap used, are you sure you have 512MB swap configured? yes - 512 MB swap > Does the installation work if you do > > echo 1 > /proc/sys/vm/overcommit_memory yes it does! Now ~350 MB of swap get used, when installing 370MB of packages. "top" shows ~350 MB res and >1.5G virt memory, but it takes ages with all the swapping. I'm wondering if rpm really needs so much memory (how much memory do you have to be able to install 11GB of packages at once???). As far as I remember, older rpm's were not so hungry... Anyway - rpm should not fail so silently and should also not be killed by OOM... Older rpms like rpm-3? yes, probably. But there were no significant changes between the versions in 10.0 and 10.1.
It actually doesn't need the memory, it just does a fork() and an exec() afterwards. The problem is that fork() duplicates the memory, that's why
setting overcommit to 1 ("ALWAYS") works.
Maybe it is a kernel problem, I'll reassign this bug to the kernel folks.
(And of course I'll change rpm to exit with a correct error message...)
Can you reproduce this with a Suse kernel, or just the mm kernel you mentioned in comment #8? (In reply to comment #14) > Can you reproduce this with a Suse kernel, or just the mm kernel you > mentioned in comment #8? > well - I'm sorry to report (or not so sorry), that this issue is no longer reproduceable after a rebuild of the rpm db. rpm now takes not more than 35MB while installing > 500MB of packages at once. I regularly rebuild my db, but this time it seemed to fix this issue which happens to me since the beginning of the 10.1 beta. I'm wondering what went wrong with the database... On the other hand, the improper behavior of rpm in low mem situtations remains, but as far as I understood, Michael will fix it. As this bug is no longer reproduceable for me, I can't test the kernel issue related to it. Many thanks and see you next time... I'm having similar trouble but with beta5. Apt-get dist-upgrade dowloads rpms pretends, but changes are not performed. rpm -U *.rpm does not remove older packages. In few cases it results with sth like: rpm -qa | grep libgcc libgcc-4.1.0_20060218-2 libgcc-4.1.0_20060223-3 I'm not sure whether it is of any interest at the moment. Should I add some more information about it please let me know. (In reply to comment #16) > I'm having similar trouble but with beta5. Apt-get dist-upgrade dowloads rpms > pretends, but changes are not performed. rpm -U *.rpm does not remove older > packages. In few cases it results with sth like: > > rpm -qa | grep libgcc > libgcc-4.1.0_20060218-2 > libgcc-4.1.0_20060223-3 > > I'm not sure whether it is of any interest at the moment. Should I add some > more information about it please let me know. I have to add, that the bug didn't went away by rebuilding the database, but by a new rpm version (4.4.2-20, beta6 I guess - I'm using factory). The changelog mentions another bug "fix cursor leak in rpmdbGrowIterator [#151953]" which sounds like a memleak to me. Unfortunately, this bug is non public (why is this needed????) You should install this single new rpm version and report back if it still happens. |