Bug 1211566

Summary: [Build 20230519] gdal failed to rebuild with zstd cmake files (identified after retried build for poppler)
Product: [openSUSE] openSUSE Tumbleweed Reporter: Dominique Leuenberger <dimstar>
Component: OtherAssignee: Bernhard Wiedemann <bwiedemann>
Status: RESOLVED FIXED QA Contact: E-mail List <qa-bugs>
Severity: Normal    
Priority: P5 - None CC: bruno, bwiedemann, code, jengelh, stefan.bruens
Version: Current   
Target Milestone: ---   
Hardware: Other   
OS: Other   
URL: https://openqa.opensuse.org/tests/3303025/modules/gdal/steps/3
Whiteboard:
Found By: openQA Services Priority:
Business Priority: Blocker: Yes
Marketing QA Status: --- IT Deployment: ---

Description Dominique Leuenberger 2023-05-20 05:50:15 UTC
## Observation

Problem: nothing provides 'libpoppler.so.127()(64bit)' needed by the to be installed libgdal32-3.6.3-1.3.x86_64
 Solution 1: do not install gdal-3.6.3-1.3.x86_64
 Solution 2: break libgdal32-3.6.3-1.3.x86_64 by ignoring some of its dependencies


openQA test in scenario opensuse-Tumbleweed-DVD-x86_64-extra_tests_textmode@64bit fails in
[gdal](https://openqa.opensuse.org/tests/3303025/modules/gdal/steps/3)

## Test suite description
Maintainer: slindomansilla@suse.de.
Mainly post-installation console extra tests.


## Reproducible

Fails since (at least) Build [20230509](https://openqa.opensuse.org/tests/3280468)


## Expected result

Last good: [20230509](https://openqa.opensuse.org/tests/3279101) (or more recent)


## Further details

Always latest result in this scenario: [latest](https://openqa.opensuse.org/tests/latest?arch=x86_64&distri=opensuse&flavor=DVD&machine=64bit&test=extra_tests_textmode&version=Tumbleweed)
Comment 1 Dominique Leuenberger 2023-05-22 11:55:34 UTC
This seems to have failed with the latest zstd 'fix' about shipping cmake files

osc rq show -d 1085997
+- Add cmake files manually because we do not want to add
+  cmake to the bootstrap ring0


gdal fails the build with
[   81s] CMake Error at /usr/lib64/cmake/zstd/zstdTargets.cmake:96 (message):
[   81s]   The imported target "zstd::libzstd_static" references the file
[   81s] 
[   81s]      "/usr/lib64/libzstd.a"
[   81s] 
[   81s]   but this file does not exist.  Possible reasons include:
[   81s] 
[   81s]   * The file was deleted, renamed, or moved to another location.
[   81s] 
[   81s]   * An install or uninstall procedure did not complete successfully.
[   81s] 
[   81s]   * The installation package was faulty and contained
[   81s] 
[   81s]      "/usr/lib64/cmake/zstd/zstdTargets.cmake"
[   81s] 
[   81s]   but not all the files it references.
[   81s] 
[   81s] Call Stack (most recent call first):
[   81s]   /usr/lib64/cmake/zstd/zstdConfig.cmake:1 (include)
[   81s]   cmake/helpers/CheckDependentLibraries.cmake:137 (find_package)
[   81s]   cmake/helpers/CheckDependentLibraries.cmake:376 (gdal_check_package)
[   81s]   gdal.cmake:264 (include)
[   81s]   CMakeLists.txt:224 (include)
Comment 2 Martin Pluskal 2023-05-22 13:53:45 UTC
(In reply to Dominique Leuenberger from comment #1)
> This seems to have failed with the latest zstd 'fix' about shipping cmake
> files
> 
> osc rq show -d 1085997
> +- Add cmake files manually because we do not want to add
> +  cmake to the bootstrap ring0
> 
> 
> gdal fails the build with
> [   81s] CMake Error at /usr/lib64/cmake/zstd/zstdTargets.cmake:96 (message):
> [   81s]   The imported target "zstd::libzstd_static" references the file
> [   81s] 
> [   81s]      "/usr/lib64/libzstd.a"
> [   81s] 
> [   81s]   but this file does not exist.  Possible reasons include:
> [   81s] 
> [   81s]   * The file was deleted, renamed, or moved to another location.
> [   81s] 
> [   81s]   * An install or uninstall procedure did not complete successfully.
> [   81s] 
> [   81s]   * The installation package was faulty and contained
> [   81s] 
> [   81s]      "/usr/lib64/cmake/zstd/zstdTargets.cmake"
> [   81s] 
> [   81s]   but not all the files it references.
> [   81s] 
> [   81s] Call Stack (most recent call first):
> [   81s]   /usr/lib64/cmake/zstd/zstdConfig.cmake:1 (include)
> [   81s]   cmake/helpers/CheckDependentLibraries.cmake:137 (find_package)
> [   81s]   cmake/helpers/CheckDependentLibraries.cmake:376
> (gdal_check_package)
> [   81s]   gdal.cmake:264 (include)
> [   81s]   CMakeLists.txt:224 (include)

I guess shipping cmake file that is refering to static library is not the best idea:


from zstdTargets-none.cmake
# Import target "zstd::libzstd_static" for configuration "None"
set_property(TARGET zstd::libzstd_static APPEND PROPERTY IMPORTED_CONFIGURATIONS NONE)
set_target_properties(zstd::libzstd_static PROPERTIES
  IMPORTED_LINK_INTERFACE_LANGUAGES_NONE "C"
  IMPORTED_LOCATION_NONE "${_IMPORT_PREFIX}/lib64/libzstd.a"
  )

I dont think gdal is at fault here
Comment 3 Stefan BrĂ¼ns 2023-05-22 21:30:02 UTC
The error is due to both targets (zstd::libzstd_static and zstd::libzstd_shared) being part of the same export set.

CMake supports multiple export sets (e.g. for shared and static libraries, non-mandatory auxiliary build tools, ...), but unfortunately many projects don't get this right - some even export tools and samples, i.e. forcing installation of the all built packages from the devel project.
Comment 4 Jan Engelhardt 2023-05-23 10:33:37 UTC
If gdal worked with previous zstd cmake files, doesn't matter if gdal got it wrong and it backfires now - introduce cmake to Hyrum's law :)
Comment 5 Bruno Friedmann 2023-05-24 06:29:35 UTC
On Holidays for a few weeks, with limited access, can't do a lot.
But maybe it is worth testing 3.6.4 upstream author's Even being quite responsive on newer lib and tools troubleshooting.
Maybe the fix is already there.
Comment 6 Benjamin Greiner 2023-05-25 11:33:56 UTC
> If gdal worked with previous zstd cmake files, doesn't matter if gdal got it
> wrong and it backfires now - introduce cmake to Hyrum's law :)

Actually gdal gets it right with their own cmake files but the introduction of zstd's cmake files in sr#1085997 overrides gdal's own cmake finder.

Upstream even knows about the problems with ztd's cmake files and works around it for conda: https://github.com/OSGeo/gdal/pull/5437

The quickfix would be to just have libzstd-devel-static in the build requirements of gdal so that the configuration stage of cmake does not fail even if there is no static linking later on. But I am very hesitant to submit that change.
Comment 7 Benjamin Greiner 2023-05-27 10:10:41 UTC
https://build.opensuse.org/request/show/1089288
Comment 8 Dominique Leuenberger 2023-10-12 13:02:22 UTC
(In reply to Benjamin Greiner from comment #7)
> https://build.opensuse.org/request/show/1089288

accepted -> fixed