systemd never supported pm-utils (because before version 195, systemd wasn't handling suspend / hibernate).

I guess pm-utils maintainer should help in migrating the quirks to systemd way of handling stuff (and we should try as much as possible to move those quirks upstream so they become unneeded).

(In reply to comment #1)
> systemd never supported pm-utils (because before version 195, systemd wasn't
> handling suspend / hibernate).

Is it possible to turn it off until this is fixed? Otherwise you'll see tons of these reports... (Depends on how fast we are at adding support for pm-utils to systemd.)

> I guess pm-utils maintainer should help in migrating the quirks to systemd way
> of handling stuff (and we should try as much as possible to move those quirks
> upstream so they become unneeded).

This is a good thing to do for the SW quirks, but mostly the quirks are rather HW bug workarounds...

(In reply to comment #2)
> (In reply to comment #1)
> > systemd never supported pm-utils (because before version 195, systemd wasn't
> > handling suspend / hibernate).
> 
> Is it possible to turn it off until this is fixed? Otherwise you'll see tons of
> these reports... (Depends on how fast we are at adding support for pm-utils to
> systemd.)

No sure. I have to investigate if I can just "plug" pm-utils into systemd hooks.

> > I guess pm-utils maintainer should help in migrating the quirks to systemd way
> > of handling stuff (and we should try as much as possible to move those quirks
> > upstream so they become unneeded).
> 
> This is a good thing to do for the SW quirks, but mostly the quirks are rather
> HW bug workarounds...

kernel should be fixed in that case ;)

Could you list exactly which quirks are still needed for you ?

(In reply to comment #4)
> Could you list exactly which quirks are still needed for you ?

On my desktop, I only need this one:
https://github.com/jirislaby/slaby-scripts/blob/master/bin/01dvb

This is to prevent suspend whenever a dvb device is active. It would lead to deadlock otherwise. I contacted DVB people about this several times, DVB layer has to be redesigned according to them.

On my notebook, I use the above plus suspend package hibernation image compression.

for dvb, it might be better to rely on systemd-inhibit(1) to prevent suspend.

(In reply to comment #6)
> for dvb, it might be better to rely on systemd-inhibit(1) to prevent suspend.

How would you do that? It might be used from whatever application at any time, not only during the runtime of some application. E.g. I have kaffeine run at the system boot, being idle. But I schedule recordings only to particular times. At those times, while /dev/dvb/* is open, suspend should be inhibited. Other than that, sometimes I use tzap, sometimes mplayer to open /dev/dvb/*. All those should somehow inhibit suspend, yes. But we are far from that. Until this is fixed, we have to use the poor hacks presented above.

The best we can do at this time is to return -EBUSY from suspend routine of the dvb drivers. But the last time I tried, I failed. The DVB layer is a mess.

(In reply to comment #7)
> (In reply to comment #6)
> > for dvb, it might be better to rely on systemd-inhibit(1) to prevent suspend.
> 
> How would you do that? It might be used from whatever application at any time,
> not only during the runtime of some application. E.g. I have kaffeine run at
> the system boot, being idle. But I schedule recordings only to particular
> times. At those times, while /dev/dvb/* is open, suspend should be inhibited.
> Other than that, sometimes I use tzap, sometimes mplayer to open /dev/dvb/*.
> All those should somehow inhibit suspend, yes. But we are far from that. Until
> this is fixed, we have to use the poor hacks presented above.

Since applications have the knowledge, they should tell logind (through D-Bus or systemd-inhibit) to present suspend / hibernation.

> I guess pm-utils maintainer should help in migrating the quirks to systemd way
> of handling stuff (and we should try as much as possible to move those quirks
> upstream so they become unneeded).

All right, could you point me to some info about that systemd way? Then I can help.

Also we have to decide which quirks we need.

How will systemd support hibernating to compressed and encrypted images?

And last but not least, we cannot just leave it broken until all this stuff is done! Even if this is just factory. Frederic, could you please make those systemd services use pm-utils for now?

(In reply to comment #9)
> > I guess pm-utils maintainer should help in migrating the quirks to systemd way
> > of handling stuff (and we should try as much as possible to move those quirks
> > upstream so they become unneeded).
> 
> All right, could you point me to some info about that systemd way? Then I can
> help.

there is no real documentation for this part yet but from http://cgit.freedesktop.org/systemd/systemd/tree/src/sleep/sleep.c , the idea is :
- hooks (is every executable in /usr/lib/systemd/system-sleep) are called before suspend / hibernate with "suspend" or "hibernate" or "hybrid-sleep"
- if hibernate or hybrid-sleep is asked:
 platform is written on /sys/power/disk (and on error, shutdown is written there
- then "mem" is written on /sys/power/state if suspend, otherwise "disk" is writtent
- when the system is resumed, the same hooks are called, with "post" as argument.
 
> Also we have to decide which quirks we need.
> 
> How will systemd support hibernating to compressed and encrypted images?

It doesn't (and I don't know if it is planned).

> And last but not least, we cannot just leave it broken until all this stuff is
> done! Even if this is just factory. Frederic, could you please make those
> systemd services use pm-utils for now?

I don't know if it can be done easily.. Maybe we could just plug pm-utils call into systemd-hibernate.service / systemd-sleep.service (but I would prefer to have the last echo "mem / disk" done by systemd.

(In reply to comment #10)
> - when the system is resumed, the same hooks are called, with "post" as
> argument.

This won't work unless there is post-disk and post-mem. You may want to do (and we do for video restore) different things on s2ram and s2disk. Maybe I just misunderstood you?

> > Also we have to decide which quirks we need.
> > 
> > How will systemd support hibernating to compressed and encrypted images?
> 
> It doesn't (and I don't know if it is planned).

Then there has to be an ability to call raw pm-utils hooks.

> (but I would prefer to have the last echo "mem / disk" done by systemd.

This won't work. s2disk has to issue the final ioctl when saving compressed or encrypted image because the rest of the world is frozen...

I like the idea to drop everything and then think how to reintroduce the wheel and construct it in a broken manner (as is presented for example by the /usr/lib/systemd/system-sleep hooks).

And I DETEST the idea of eating separate libraries (udev, syslog, or pm-utils in this case) and screw them up terribly, because they have _no_ clue what the libraries were about.

(In reply to comment #11)
> (In reply to comment #10)
> > - when the system is resumed, the same hooks are called, with "post" as
> > argument.
> 
> This won't work unless there is post-disk and post-mem. You may want to do (and
> we do for video restore) different things on s2ram and s2disk. Maybe I just
> misunderstood you?

Well, maybe we could add the "verb" used (disk / mem) as second argument. I'm was just describing the code I linked in my previous comment ;)

Hmm, this raises another question...

How did systemd plan to support splash during hibernation/resume? This is exactly what suspend package called from pm-utils did. (But I never used that.)

So any news on this? Frederic, do you have some kind of a plan? Does suspending and hibernating work in 12.3 beta?

no plan (I'm really short on time). Suspend and hibernation works in 12.3 beta, the issue is for pm-utils "quirks" and scripts, which are not run.

(In reply to comment #15)
> no plan (I'm really short on time). Suspend and hibernation works in 12.3 beta,
> the issue is for pm-utils "quirks" and scripts, which are not run.

Michael, how does this affect grub+hibernation? We used to generate some configuration during freeze phase for grub...

And we check in the scripts whether we still have a kernel to boot the hibernated from and forbid if we don't. Is this done in systemd path to hibernation?

Although the boot and image kernels can differ on x86_64, other arches do not support this. They need exactly the same kernel to restore the one which was hibernated. So how exactly does systemd take care of not losing user's data in this case?

(In reply to comment #17)
> And we check in the scripts whether we still have a kernel to boot the
> hibernated from and forbid if we don't. Is this done in systemd path to
> hibernation?
> 
> Although the boot and image kernels can differ on x86_64, other arches do not
> support this. They need exactly the same kernel to restore the one which was
> hibernated. So how exactly does systemd take care of not losing user's data in
> this case?

I'm not aware of such check in systemd but I guess can_sleep / can_sleep_disk in http://cgit.freedesktop.org/systemd/systemd/tree/src/shared/util.c could be extended (but it wouldn't be trivial in C). It might be worth discussing it with upstream.

(In reply to comment #16)
> (In reply to comment #15)
> > no plan (I'm really short on time). Suspend and hibernation works in 12.3 beta,
> > the issue is for pm-utils "quirks" and scripts, which are not run.
> 
> Michael, how does this affect grub+hibernation? We used to generate some
> configuration during freeze phase for grub...

Grub(or any bootloader) is unaware of hibernation and those hooks are necessary to have bootloader configs properly set when resuming, that is to boot the exact hibernated kernel entry without any user's intervention. 

Any user's intervention is unneeded, bad user experience and probably ruined your data, consider the case that another one takes your hibernated notebook and boot, he has no idea but casually boots his favorite distribution with shared swap ... no one can really tell what will happen next.

Btw we have both grub and grub2 pm hooks for hibernation.

(In reply to comment #19)
> (In reply to comment #16)
> > (In reply to comment #15)
> > > no plan (I'm really short on time). Suspend and hibernation works in 12.3 beta,
> > > the issue is for pm-utils "quirks" and scripts, which are not run.
> > 
> > Michael, how does this affect grub+hibernation? We used to generate some
> > configuration during freeze phase for grub...
> 
> Grub(or any bootloader) is unaware of hibernation and those hooks are necessary
> to have bootloader configs properly set when resuming, that is to boot the
> exact hibernated kernel entry without any user's intervention. 
> 
> Any user's intervention is unneeded, bad user experience and probably ruined
> your data, consider the case that another one takes your hibernated notebook
> and boot, he has no idea but casually boots his favorite distribution with
> shared swap ... no one can really tell what will happen next.

Hmm, this smells like a blocker to me. Coolo?

let's track it

I don't like this "systemd handles suspend" idea at all. It seems to me as if it only meant a lot of trouble for no gain.
I think we should keep pm-utils and let it handle suspending and hibernating. Why would we want to change it anyway?

Does the systemd approach have any plus sides? What are the arguments for switching from pm-utils to systemd apart from the abstract idea that it should be an "all in one" thing? (I'd really like to hear them, I hope this doesn't sound too ironic).

And whatever the answer is, please consider reverting to pm-utils at least for 12.3, I don't think we can create a working systemd suspend/hibernate scheme until it is released anyway.

(In reply to comment #22)
> I don't like this "systemd handles suspend" idea at all. It seems to me as if
> it only meant a lot of trouble for no gain.
> I think we should keep pm-utils and let it handle suspending and hibernating.
> Why would we want to change it anyway?

Because some developers are so "clever":
http://lists.opensuse.org/opensuse-factory/2012-11/msg00066.html
http://lists.opensuse.org/opensuse-factory/2013-01/msg00271.html
http://lists.opensuse.org/opensuse-factory/2012-11/msg00217.html

> I don't think we can create a working systemd suspend/hibernate scheme
> until it is released anyway.

Well, the fix is should not be that hard. There was an option in systemd to disable this brand new broken API. If that option is gone now, it should still not be hard to revert that change I think.

strangely enough, none of you proposed a patch for systemd which could call pm-utils when it is installed, so its hooks would be called and it would delegate suspend to pm-utils. This would a big ugly hack but I guess it would satisfy you..

We didn't break that so to speak:
http://lists.opensuse.org/opensuse-factory/2013-01/msg00268.html
http://lists.opensuse.org/opensuse-factory/2013-01/msg00275.html
http://lists.opensuse.org/opensuse-factory/2013-01/msg00285.html
http://lists.opensuse.org/opensuse-factory/2013-01/msg00259.html

To set it clear: I'm not blaming _you_. This is upstream developers who screw up things constantly and regularly and putting their fingers off when comes to fixing bugs caused by their stupid ideas.

Sorry but I have zero interest in arguing about what upstream did or should do, etc..

Feel free to discuss with upstream directly.

But for now, I still intend to keep this API (since it is being used and expected to become standard API on Linux (excluding Ubuntu)). 

I might try to patch systemd so it calls pm-utils (which will make suspend / hibernation much slower and not better, at least for me) if I have time. If some of you can cook a patch, I'll be happy to look at it and integrate it.

(In reply to comment #26)
> Sorry but I have zero interest in arguing about what upstream did or should do,
> etc..

Then please stop applying new systemd releases to openSUSE and stick to some version plus opensuse fixes. Hmm, cruel idea, in that case we are going to have a new sysvinit, divergent and distro-specific. And the same happened to grub and now it threatens to grub2 (you always need some specific hook which upstream doesn't like for whatever reason). Now I started thinking that these kind of services cannot be fully unified in the end. Blah.

> But for now, I still intend to keep this API (since it is being used and
> expected to become standard API on Linux (excluding Ubuntu)). 

What is that distribution used by majority going to do then? Sorry, it cannot become a standard unless used on major distributions.

Well, I have nothing against that API. If and only if it adapts what pm-utils do. Be it calling its hooks or re-inventing the wheel -- no matter which one.

> I might try to patch systemd so it calls pm-utils (which will make suspend /
> hibernation much slower and not better, at least for me)

Oh, "at least for me" is exactly Kay's and Lennart's argument and it is pure and utter @#$. There are gazillion of people not sitting behind their laptops. They are using diverse configurations, different hardware and different distros. Please stop using that argument, it's like I would say that the kernel works for me and I don't care about your crashes.

pm-utils is maybe slow, but it works, because it was improved and tuned for years for (let's call it) unusual hardware. I really want to have my data safe rather than having faster hibernation.

> if I have time. If
> some of you can cook a patch, I'll be happy to look at it and integrate it.

I'm not going to taint my hands with that, sorry.

(In reply to comment #27)

> > if I have time. If
> > some of you can cook a patch, I'll be happy to look at it and integrate it.
> 
> I'm not going to taint my hands with that, sorry.

Then I'll do the work myself, when I find time.

(In reply to comment #28)
> (In reply to comment #27)
> 
> > > if I have time. If
> > > some of you can cook a patch, I'll be happy to look at it and integrate it.
> > 
> > I'm not going to taint my hands with that, sorry.
> 
> Then I'll do the work myself, when I find time.

Ok, thanks.

We still can change "%define with_systemd" to 0 in upower to revert, if you run out of time, right?

> We still can change "%define with_systemd" to 0 in upower to revert, if you run
> out of time, right?

Oh please let's do that! It is easy like that to fix all those issues.

Let's summarize:

Arguments for using pm-utils:
-Supports encrypted and compressed images (systemd doesn't even plan to support
them in the future)
-Has hw workarounds
-Handles grub/grub2/grub2-efi so that it always resumes after hibernate
-Has splash (ok thats not so important)
-Works well and was thorougly tested with many bugs fixed
-Easy to implement - one change in upower spec-file vs. either rewriting all
the hooks to systemd or using an ugly hack that actually uses both and doesn't
support encrypted/compressed images anyway.

Arguments for using systemd for suspend:
-Redhat wants it to be a future standard on all main distros except Ubuntu.

Please let's just use pm-utils. Christian, you are the maintainer of upower,
are you willing to do this little change?

Frederic, will you agree?

I do not maintain upower and I disagree that we have to revert systemd support in upower, if anything we have to take the approach Frederic suggests.

(In reply to comment #30)
> Arguments for using systemd for suspend:
> -Redhat wants it to be a future standard on all main distros except Ubuntu.
> 
> Please let's just use pm-utils. Christian, you are the maintainer of upower,
> are you willing to do this little change?

Note that this is only a temporary solution. Feel free to set it back to 1 when you feel all the above should work with systemd enabled in upower and you cannot foresee any regressions based on your experience and facts mentioned in this bugreport.

(Now we have 12.3 in the list of products, so change accordingly.)

Could we please stop the conspiracy theory ?

I'd prefer to have systemd API used (because people might start using the
systemd hooks), rather than having two different ways of suspending, which will
do different things.

And I'll stop commenting, because it is taking me time which would be better
spend in doing the patch in the first place.

(In reply to comment #31)
> I disagree that we have to revert systemd support
> in upower, if anything we have to take the approach Frederic suggests.

Yes, right, we don't have to if Frederic manages to do the work. If he runs out time, we unfortunately have no other option, do we?

please do not reassign to screening team

please test package from home:fcrozat:branches:Base:System/systemd, it contains :
- support for hybrid-sleep (backported from upstream systemd)
- forward to pm-utils for hibernate, sleep and suspend-hibernate, if pm-utils tool is installed. Note that systemd own hooks are still run (and my hope would be to move slowly pm-utils hooks to systemd one).

package should be available there in about one hour max.

Yeah, this is brilliant, we're almost there. sleep crashes when some pm-utils hook inhibits suspend:
traps: systemd-sleep[24303] general protection ip:7fca4d6e1d01 sp:7ffff2f6cdf0 error:0 in libc-2.17.so[7fca4d66b000+1a3000]

That IP is in fclose. Of course, fclose is now called on unopened descriptor...

Hi Frederic!
After updating to the package (systemd) in your branch, I have the following problems:

1. Hibernate (pm-hibernate) wont work, it saves the disk image and shuts down, but during start up it logs in automatically to openSUSE (without presenting bootloader menu), which is expected; but then starts as if on a clean restart, getting me to the gdm screen. Every thing saved during the hibernation seems lost. This was working before the update.

2. The first suspend after boot has to be initiated manually (either power-button, or suspend key or pm-suspend), and it works; but attempting to do the first suspend after a boot by closing the laptop lid does not work, i.e. nothing happens. This is only about the first suspend after boot; thereafter suspend works fine also on screen-lid close. This problem was also present before updating to the packages in your branch.

If this seems like a different problem, I will be happy to open a new bug.

Thanks.

(In reply to comment #37)
> Yeah, this is brilliant, we're almost there. sleep crashes when some pm-utils
> hook inhibits suspend:
> traps: systemd-sleep[24303] general protection ip:7fca4d6e1d01 sp:7ffff2f6cdf0
> error:0 in libc-2.17.so[7fca4d66b000+1a3000]
> 
> That IP is in fclose. Of course, fclose is now called on unopened descriptor...

Indeed, I missed this one. Fix pushed to OBS, should be available in a few minutes

(In reply to comment #38)
> Hi Frederic!
> After updating to the package (systemd) in your branch, I have the following
> problems:
> 
> 1. Hibernate (pm-hibernate) wont work, it saves the disk image and shuts down,
> but during start up it logs in automatically to openSUSE (without presenting
> bootloader menu), which is expected; but then starts as if on a clean restart,
> getting me to the gdm screen. Every thing saved during the hibernation seems
> lost. This was working before the update.
> 
> 2. The first suspend after boot has to be initiated manually (either
> power-button, or suspend key or pm-suspend), and it works; but attempting to do
> the first suspend after a boot by closing the laptop lid does not work, i.e.
> nothing happens. This is only about the first suspend after boot; thereafter
> suspend works fine also on screen-lid close. This problem was also present
> before updating to the packages in your branch.
> 
> If this seems like a different problem, I will be happy to open a new bug.

I'm 99% those are different problems.

Just test by calling "pm-suspend" or "pm-hibernate" as root. If it is "broken", it is a bug in pm-utils, not in systemd.

(In reply to comment #40)
> (In reply to comment #38)
> > Hi Frederic!
> > After updating to the package (systemd) in your branch, I have the following
> > problems:
> > 
> > 1. Hibernate (pm-hibernate) wont work, it saves the disk image and shuts down,
> > but during start up it logs in automatically to openSUSE (without presenting
> > bootloader menu), which is expected; but then starts as if on a clean restart,
> > getting me to the gdm screen. Every thing saved during the hibernation seems
> > lost. This was working before the update.

This is a regression in behaviour when I update to the systemd package in your branch; indeed, when I roll-back to systemd package from the OSS repository I can hibernate and (although it shows the grub2 menu, etc at boot which it should not) it brings me back to the state I hibernated my computer in. It does not behave like a fresh boot up. With systemd packages from your branch, hibernation leads me to gdm screen, losing all saved status completely. Should I open a new bug for this too?


> 
> I'm 99% those are different problems.
> 
> Just test by calling "pm-suspend" or "pm-hibernate" as root. If it is "broken",
> it is a bug in pm-utils, not in systemd.

pm-hibernate was working fine with systemd from the OSS repository, but no longer since I updated to the branch packages as I have mentioned above. Clearly, pm-hibernate isnt broken.

Well, I'll be honest. People were complaining because systemd wasn't hooked to pm-utils. With this patch, systemd doesn't handle suspend / hibernate any more, it just calls pm-utils. So any bug (or regressions) caused by this change will have to be a separate bug and will have to be handled by pm-utils maintainer.

I've submitted the package to Factory. Closing bug as fixed.

This is an autogenerated message for OBS integration:
This bug (790157) was mentioned in
https://build.opensuse.org/request/show/155884 Factory / systemd

(In reply to comment #42)
> So any bug (or regressions) caused by this change will
> have to be a separate bug and will have to be handled by pm-utils maintainer.

And when you are creating one, don't forge to attach /var/log/pm-suspend.log. It would be good to have the bug number here. And there to have this bug number...

I have opened bnc#805875 for the hibernation problem I reported in comment 38.

(In reply to comment #42)
> Well, I'll be honest. People were complaining because systemd wasn't hooked to
> pm-utils. With this patch, systemd doesn't handle suspend / hibernate any more,
> it just calls pm-utils. So any bug (or regressions) caused by this change will
> have to be a separate bug and will have to be handled by pm-utils maintainer.
> 
> I've submitted the package to Factory. Closing bug as fixed.

OK frederic, this means all of us must use a piece of software that is abandoned since 3 years ? 

That is INSANE, sorry.

(In reply to comment #46)
> (In reply to comment #42)
> > Well, I'll be honest. People were complaining because systemd wasn't hooked to
> > pm-utils. With this patch, systemd doesn't handle suspend / hibernate any more,
> > it just calls pm-utils. So any bug (or regressions) caused by this change will
> > have to be a separate bug and will have to be handled by pm-utils maintainer.
> > 
> > I've submitted the package to Factory. Closing bug as fixed.
> 
> OK frederic, this means all of us must use a piece of software that is
> abandoned since 3 years ? 

What kind of argument is this? COBOL is discontinued for decades and is still in active use. pm-utils just work. They are not damn fast, but work.

> That is INSANE, sorry.

That is not insane, that is reality. Until systemd handles all the cases which pm-utils used to (which is the long-term plan), there is no other way. The rule you should finally remember is NO REGRESSIONS. We are not allowed to break a single box and that was not fulfilled by the change, see now?

> That is not insane, that is reality. Until systemd handles all the cases which
> pm-utils used to (which is the long-term plan), there is no other way. The rule
> you should finally remember is NO REGRESSIONS. We are not allowed to break a
> single box and that was not fulfilled by the change, see now?


systemd will not handle kernel bugs.. that's for sure, however I agree it must handle the hibernate/suspend to encrypted images.

My main concern, is that a new distribution is being made that will ship a mutant version of systemd that calls pm-utils, a piece of code that is abandoned for the sole purpose of covering a few corner cases for a few buggy hardware.
 
Your rule idea of no regressions sounds nice but it is completely unrealistic.

Anyway, released versions of upower and therefore KDE power managment still need pm-utils to be installed. that has been already fixed (in the sense pm-utils will no longer be called) but will not appear in 12.3..

Created attachment 569356 [details]
Systemd unit to be used instead of 99ZGrub

This is the grub hook, modified so it can be put to /usr/lib/systemd/system-sleep and used without pm-utils.

Hi Vojtech,

Will you submit to factory ?

Well I would prefer to do some more stuff and then submit it all at once. Why do you think should I submit it now? Currently pm-utils do it anyway.
If you have any suggestions, please send them to the factory mailing list, that's where we discuss the future dropping of pm-utils.