Bug 1120035

Summary: postgresql11 has files in server-devel that are needed for building client applications
Product: [openSUSE] openSUSE Tumbleweed Reporter: Christian Ehrlicher <ch.ehrlicher>
Component: OtherAssignee: Reinhard Max <max>
Status: RESOLVED FIXED QA Contact: E-mail List <qa-bugs>
Severity: Normal    
Priority: P5 - None CC: bruno, ch.ehrlicher, josef.moellers, max, ro, stefan.bruens
Version: Current   
Target Milestone: ---   
Hardware: Other   
OS: Other   
Whiteboard:
Found By: --- Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---

Description Christian Ehrlicher 2018-12-20 07:38:04 UTC
postgresql11-devel is missing some files so it's unusable for now:
e.g.
bin/pg_config
include/pqsql/postgres_ext.h

They seem to be in postgresql11-server-devel but I just want to lbuild/ink against postgresql client libs, not server (e.g. when compiling Qt).
Comment 1 Reinhard Max 2019-01-11 11:20:16 UTC
pg_config contains only information about the server, so it does belong to the server-devel package. Applications that want to link against client libraries should use pkg-config to query details about the respective client library they use.

Regarding include/pqsql/postgres_ext.h you might have a valid point. I will investigate that.
Comment 2 Reinhard Max 2019-01-11 13:30:25 UTC
Fixed:

-------------------------------------------------------------------
Fri Jan 11 13:23:20 UTC 2019 - Reinhard Max <max@suse.com>

- Adjust the line between headers going to devel and server-devel.
  Now everything under /usr/include/pgsql/server is in server-devel
  and the rest of /usr/include/pgsql is in devel (bsc#1120035).
-------------------------------------------------------------------

But please use a more descriptive summary line next time you report a bug.
Just calling it "Bug" (even together with the package name) doesn't say much.
Comment 3 Christian Ehrlicher 2019-01-11 14:57:01 UTC
This came from https://build.opensuse.org/package/show/server:database:postgresql/postgresql11 -> Report Bug which automatically fills this line and I forgot to modify it.
Comment 4 Stefan BrĂ¼ns 2019-03-15 19:29:19 UTC
(In reply to Reinhard Max from comment #1)
> pg_config contains only information about the server, so it does belong to
> the server-devel package. 

So why does it have e.g.:

--includedir
    Print the location of C header files of the client interfaces.
--pkgincludedir
    Print the location of other C header files.
--includedir-server
    Print the location of C header files for server programming.

https://www.postgresql.org/docs/11/app-pgconfig.html
Comment 5 Reinhard Max 2019-03-21 08:33:13 UTC
*** Bug 1130021 has been marked as a duplicate of this bug. ***
Comment 6 Reinhard Max 2019-03-21 09:12:39 UTC
OK, I see that this needs a much deeper explanation.

In the old days of PostgreSQL the configure script collected all the libs that were needed for the backend and the frontend and put them into a single LIBS variable that was used for both sides. This caused the server to be linked against libs that were only relevant for the client side and vice versa. This list was also compiled into pg_config and queryable with --libs.

In the meantime this has been cleaned up and today PostgreSQL has separate lists for clients and servers. Also, support for pkg-config was added for libpq and libecpg and I think that is what should be used instead of pg_config these days.

The rest of the story has to do with the way we compile PostgreSQL when rebuilding the distro:

On the one hand we have to buid PostgreSQL early in the bootstrapping process, because libpq is needed by some large chunks like Qt (and hence KDE), Python, etc. On the other hand, the PostreSQL server itself needs Python, Perl and Tcl for building the respective procedudural language modules.

That's why we have split building PostgreSQL into two stages, one that has only small dependencies and builds just the client libraries and the embedded SQL preprocessor and one that builds the rest and depends on stuff that is only relevant for the server.

@Rudi Can you confirm that this split is still needed and allows for way better parallelism when doing full distro-rebuilds?

Up to PosrgreSQL 10 I used some tricks to get pg_config built in the first stage and packaged the header files for the frontend and the backend into postgresqlXX-devel. In hindsight this was not correct, because the so-built pg_config was missing the additional information that was only available when actually building the backend.

With version 11 upstream changed and cleaned up the build system of PostgreSQL quite a bit, which made it much harder to get pg_config built without also building the whole backend. That's why I decided to split the devel package between client and server and pg_config naturally ended up on the server side.

If now packages that want to link agains the client libs pull in server-devel this defeats the whole purpose of the split build of the backend and frontend part of PostgreSQL, because such packages have to wait until the backend (and all its dependencies) have been built.

I think instead all client packages that currently use pg_config should be changed to use pkg-config instead. I've done that already for a few packages that broke while PostgreSQL 11 was in staging. Some of them already use pkg-config for other libs, so they shouldn't be hard to change.

If anybody has an idea for a better solution, I am all ears.
Comment 7 Christian Ehrlicher 2019-03-21 09:35:33 UTC
I still think that include/pqsql/postgres_ext.h is needed when trying to compile the Qt5 postgres driver. Will take a look soon.
Comment 8 Reinhard Max 2019-03-21 09:47:09 UTC
(In reply to Christian Ehrlicher from comment #7)

> I still think that include/pqsql/postgres_ext.h is needed when trying to
> compile the Qt5 postgres driver. Will take a look soon.

I fixed this already, but for some reason the automatism for posting bug references in SRs to the respective bugs didn't work in this case.

-------------------------------------------------------------------
Fri Jan 11 13:23:20 UTC 2019 - Reinhard Max <max@suse.com>
[...]
- Adjust the line between headers going to devel and server-devel.
  Now everything under /usr/include/pgsql/server is in server-devel
  and the rest of /usr/include/pgsql is in devel (bsc#1120035).

-------------------------------------------------------------------
Comment 9 Bruno Friedmann 2019-03-21 10:31:35 UTC
@Reinhard thanks for the deep explanation, which thus make sense. 

I'm just complaining that this part of information was missing in the .changes 
I will try to adapt the packages I'm maintaining that should only need the libpq.

If all of that happened few month ago, we would have been a bit less understress with next Leap release so close ;-)
Comment 10 Bruno Friedmann 2019-03-21 10:37:14 UTC
Now I'm still clueless about how to fix packages that used

pg_config --libdir 

/usr/lib/postgresql10/lib64

there's no pkg-config able to give the right path.

pkg-config libpq --libs
give only -lpq 
I understand that I can add -L%{_libdir} but that one will higly depend of alternatives which I estimate a bit fragile.
Comment 11 Reinhard Max 2019-03-21 11:07:18 UTC
(In reply to Bruno Friedmann from comment #10)
> Now I'm still clueless about how to fix packages that used
> 
> pg_config --libdir 
> 
> /usr/lib/postgresql10/lib64

we only have shared objects there that are related to the backend, but install the client libraries under %_libdir as we always only provide one version of the libraries (the latest), multiple exclusive versions of the devel packages and multiple cuncurrently installable versions of the actual binaries.

> there's no pkg-config able to give the right path.
> 
> pkg-config libpq --libs
> give only -lpq 

That's enough, you don't need anything else.

> I understand that I can add -L%{_libdir} but that one will higly depend of
> alternatives which I estimate a bit fragile.

Alternatives are not involved here as only one version of the devel packages can be installed at the same time.

If that's seen as being too confusing, I could change the devel packages to use alternatives as well and have the respective -L-Flag in --libs. Actually I am currently patching that piece out of the pkg-config files, because it becomes wrong and unneeded when I move the libs to %_libdir.
Comment 12 Ruediger Oertel 2019-03-21 12:43:49 UTC
@Rudi Can you confirm that this split is still needed and allows for way better parallelism when doing full distro-rebuilds?

I'm completely out of the game with tracking dependency graphs these days,
you'd have to ask coolo or ludwig about this. but in general these dependencies do only get worse over time and never better (at least I
can't even remember a single case).
Comment 13 Bruno Friedmann 2019-03-22 07:01:39 UTC
I've learn today when fixing sphinxsearch that there's a subtile difference

pg_config --includedir 
 /usr/include/pgsql -lpq 

when 
pkg-config --cflags --libs libpq 
 -I/usr/include/pgsql -lpq

I had to sed remove the -I to make software happy (otherwise I've got -I-I/usr/include/pgsql
Comment 14 Reinhard Max 2019-12-04 15:51:36 UTC
I guess we can close, as the placement of postgres_ext.h has been adjusted and packages using pg_config probably got converted to use pkg-config by now (or maintainers went the discouraged albeit easier way of requiring the server-devel package).