Apache 2.4.51 Error: Cannot load libexec/apache24

Yesterday I upgraded our WebDAV host (iocage jail) to 12.2.p10 and the apache24 software to 2.4.51 via pkg. Both the jail and the service restarted without error following the upgrades.

This morning the WebDAV resource was reported as unreachable. Investigation showed that the apache24 service on that host was stopped. Running service apache24 configtest reported OK. Restarting the service was successful with no errors reported.

The main error log contained this entry:
Code:
Fri Oct 15 00:00:01.149902 2021] [mpm_prefork:notice] [pid 53755] AH00171: Graceful restart requested, doing restart
httpd: Syntax error on line 23 of /usr/local/etc/apache24/httpd.conf:
Cannot load libexec/apache24/mod_dav.so into server: /usr/local/libexec/apache24/mod_dav.so:
 Undefined symbol "ap_parse_strict_length"

The entries immediately after this one were those concerned with starting the service later that morning.

The referenced file exists and has a recent date so I infer that this was created by the recent upgrade to Apache.
Code:
# ll /usr/local/libexec/apache24/mod_dav.so
-rwxr-xr-x  1 root  wheel  107304 Oct  9 21:12 /usr/local/libexec/apache24/mod_dav.so

The reported undefined symbol seems to be present in that file:
Code:
# strings  /usr/local/libexec/apache24/mod_dav.so | grep -i ap_parse_strict_length
ap_parse_strict_length

The trigger for this appears to be the nightly log rotation by newsyslog. The questions are: What is causing this error. How is it fixed?
 
What does nm -D /usr/local/libexec/apache24/mod_dav.so give you? Is the method in the exported functions list?
Have you done a ldd /usr/local/libexec/apache24/mod_dav.so and checked for any missing dependencies?

I assume, you did explicitly stop and terminate all httpd processes, since you wrote you restarted the jail? For obvious reasons, just doing a reload after a library upgrade and not doing an explicit stop/start will lead to unexpected behavior.
 
use nm -D
Code:
nm -D /usr/local/sbin/httpd |grep ap_parse_strict_length
00000000002a9f20 D ap_hack_ap_parse_strict_length
000000000026b9f0 T ap_parse_strict_length

nm -D /usr/local/libexec/apache24/mod_dav.so |grep ap_parse_strict_length
U ap_parse_strict_length
should be exported by httpd
 
What does nm -D /usr/local/libexec/apache24/mod_dav.so give you? Is the method in the exported functions list?
Have you done a ldd /usr/local/libexec/apache24/mod_dav.so and checked for any missing dependencies?

I assume, you did explicitly stop and terminate all httpd processes, since you wrote you restarted the jail? For obvious reasons, just doing a reload after a library upgrade and not doing an explicit stop/start will lead to unexpected behavior.
Code:
[root@webdav ~]#  nm -D /usr/local/libexec/apache24/mod_dav.so
                 U _CurrentRuneLocale
                 w _Jv_RegisterClasses
                 U _ThreadRuneLocale
                 w __cxa_finalize
                 U __mb_sb_limit
                 U __stack_chk_fail
                 U __stack_chk_guard
                 U __tls_get_addr
000000000001977c T _fini
0000000000019768 T _init
                 U ap_construct_url
                 U ap_cstr_casecmp
                 U ap_cstr_casecmpn
                 U ap_destroy_sub_req
                 U ap_discard_request_body
                 U ap_escape_html2
                 U ap_filter_flush
                 U ap_fprintf
                 U ap_fputstrs
                 U ap_get_brigade
                 U ap_get_status_line
                 U ap_getword_white
                 U ap_hook_fixups
                 U ap_hook_handler
                 U ap_hook_post_config
                 U ap_list_provider_names
                 U ap_log_error_
                 U ap_log_rerror_
                 U ap_lookup_provider
                 U ap_map_http_request_error
                 U ap_meets_conditions
                 U ap_method_register
                 U ap_normalize_path
                 U ap_os_escape_path
                 U ap_parse_strict_length
                 U ap_pass_brigade
                 U ap_psignature
                 U ap_register_provider
                 U ap_rprintf
                 U ap_run_http_scheme
                 U ap_rvputs
                 U ap_rwrite
                 U ap_set_content_length
                 U ap_set_content_type
                 U ap_sub_req_lookup_uri
                 U ap_sub_req_method_uri
                 U ap_unescape_url
                 U ap_xml_parse_input
                 U apr_array_make
                 U apr_array_push
                 U apr_brigade_cleanup
                 U apr_brigade_create
                 U apr_brigade_destroy
                 U apr_brigade_putc
                 U apr_brigade_write
                 U apr_bucket_eos_create
                 U apr_bucket_type_eos
                 U apr_hash_first
                 U apr_hash_get
                 U apr_hash_make
                 U apr_hash_next
                 U apr_hash_set
                 U apr_hash_this
                 U apr_hook_debug_current
                 U apr_hook_debug_enabled
                 U apr_hook_debug_show
                 U apr_hook_global_pool
                 U apr_hook_sort_register
                 U apr_palloc
                 U apr_pool_cleanup_null
                 U apr_pool_cleanup_register
                 U apr_pool_clear
                 U apr_pool_create_ex
                 U apr_pool_destroy
                 U apr_pool_tag
                 U apr_psprintf
                 U apr_pstrcat
                 U apr_pstrdup
                 U apr_pstrmemdup
                 U apr_snprintf
                 U apr_table_addn
                 U apr_table_elts
                 U apr_table_get
                 U apr_table_make
                 U apr_table_set
                 U apr_table_setn
                 U apr_table_unset
                 U apr_text_append
                 U apr_uri_parse
                 U apr_uri_port_of_scheme
                 U apr_uri_unparse
                 U apr_xml_empty_elem
                 U apr_xml_quote_elem
                 U apr_xml_quote_string
                 U apr_xml_to_text
                 U atoi
                 U atol
                 U bcmp
0000000000018fd0 T dav_add_all_liveprop_xmlns
0000000000018670 T dav_add_lock
000000000000b670 T dav_add_response
0000000000017840 T dav_add_vary_header
0000000000017e50 T dav_auto_checkin
00000000000178d0 T dav_auto_checkout
000000000000afa0 T dav_begin_multistatus
0000000000015730 T dav_buffer_append
00000000000156c0 T dav_buffer_init
00000000000157d0 T dav_buffer_place
0000000000015860 T dav_buffer_place_mem
0000000000015630 T dav_check_bufsize
000000000000b660 T dav_close_lockdb
0000000000014520 T dav_close_propdb
0000000000019340 T dav_core_find_liveprop
0000000000019360 T dav_core_insert_all_liveprops
00000000000196b0 T dav_core_register_uris
0000000000019090 T dav_do_find_liveprop
000000000000b6e0 T dav_failed_proppatch
0000000000015af0 T dav_find_child
000000000000b060 T dav_finish_multistatus
0000000000014560 T dav_get_allprops
000000000000ad20 T dav_get_binding_hooks
000000000000b420 T dav_get_depth
0000000000019140 T dav_get_liveprop_info
0000000000018fc0 T dav_get_liveprop_ns_count
0000000000018fa0 T dav_get_liveprop_ns_index
0000000000014f80 T dav_get_liveprop_supported
000000000000ac90 T dav_get_lock_hooks
0000000000017720 T dav_get_locktoken_list
00000000000192e0 T dav_get_options_providers
000000000000acc0 T dav_get_propdb_hooks
0000000000014b00 T dav_get_props
000000000000ac70 T dav_get_provider
000000000000ac40 T dav_get_provider_name
000000000000b4f0 T dav_get_resource
0000000000018b20 T dav_get_resource_state
0000000000019320 T dav_get_resource_type_providers
000000000000ad50 T dav_get_search_hooks
0000000000016020 T dav_get_timeout
000000000000acf0 T dav_get_vsn_hooks
000000000000b200 T dav_handle_err
000000000000c150 T dav_hook_deliver_report
000000000000bed0 T dav_hook_find_liveprop
000000000000bdb0 T dav_hook_gather_propsets
000000000000c2a0 T dav_hook_gather_reports
000000000000c200 T dav_hook_get_deliver_report
000000000000bf80 T dav_hook_get_find_liveprop
000000000000be60 T dav_hook_get_gather_propsets
000000000000c350 T dav_hook_get_gather_reports
000000000000c0c0 T dav_hook_get_insert_all_liveprops
000000000000c490 T dav_hook_get_method_precondition
000000000000c010 T dav_hook_insert_all_liveprops
000000000000c3e0 T dav_hook_method_precondition
00000000000155f0 T dav_join_error
0000000000018160 T dav_lock_get_activelock
00000000000183f0 T dav_lock_parse_lockinfo
0000000000018860 T dav_lock_query
00000000000192a0 T dav_lookup_provider
00000000000158e0 T dav_lookup_uri
000000000001c4f8 D dav_module
00000000000154e0 T dav_new_error
0000000000015530 T dav_new_error_tag
0000000000018c30 T dav_notify_created
000000000000b620 T dav_open_lockdb
0000000000014380 T dav_open_propdb
00000000000192c0 T dav_options_provider_register
0000000000014450 T dav_popen_propdb
0000000000015410 T dav_prop_commit
00000000000152e0 T dav_prop_exec
0000000000015450 T dav_prop_rollback
00000000000150d0 T dav_prop_validate
00000000000155a0 T dav_push_error
0000000000019190 T dav_register_liveprop_group
0000000000019280 T dav_register_provider
0000000000019300 T dav_resource_type_provider_register
000000000000c210 T dav_run_deliver_report
000000000000bf90 T dav_run_find_liveprop
000000000000be70 T dav_run_gather_propsets
000000000000c360 T dav_run_gather_reports
000000000000c0d0 T dav_run_insert_all_liveprops
000000000000c4a0 T dav_run_method_precondition
000000000000b0d0 T dav_send_multistatus
000000000000ad80 T dav_send_one_response
0000000000015680 T dav_set_bufsize
000000000000b8d0 T dav_success_proppatch
0000000000018890 T dav_unlock
0000000000016110 T dav_validate_request
0000000000015ac0 T dav_validate_root
0000000000015b40 T dav_xml_get_cdata
0000000000015e30 T dav_xmlns_add
0000000000015e80 T dav_xmlns_add_uri
0000000000015de0 T dav_xmlns_create
0000000000015f50 T dav_xmlns_generate
0000000000015f30 T dav_xmlns_get_prefix
0000000000015f10 T dav_xmlns_get_uri
                 U memcpy
                 U memset
                 U strcasecmp
                 U strchr
                 U strcmp
                 U strcpy
                 U strlen
                 U strncmp
                 U strrchr
                 U time
Code:
[root@webdav ~]# ldd /usr/local/libexec/apache24/mod_dav.so
/usr/local/libexec/apache24/mod_dav.so:
    libc.so.7 => /lib/libc.so.7 (0x80024e000)
[root@webdav ~]# ll /lib/libc.so.7
-r--r--r--  1 root  wheel  1952200 Oct 14 08:27 /lib/libc.so.7
 
Well, this tells us the method is undefined in the library.

I assume you have installed apache 2.4.51, the most recent version according to freshports. What does httpd -v (or whatever makes apache print its version) tell you? Does that show the same version as the package?
 
I am providing what you ask as soon as I can. However, at the moment I have another situation consuming my time so I unable to provide more than the bare bones by way of reply.

Code:
[root@webdav ~]# httpd -v
Server version: Apache/2.4.51 (FreeBSD)
Server built:   unknown
 
Strange. I'm no apache expert - from what I can read in the docs though, that method is likely defined in apache's core and should be there.
One possible reason for the symptoms you're seeing might be version mismatches. I'd try to service apache stop, check with fuser whether anything is still touching httpd or mod_dav.so, kill any remaining processes of httpd and then start it up again - once there's nothing else having a file handle to the lib or the apache executable.

Then again, maybe you've tried that already?
 
If I read this aright then is not ap_parse_strict_length defined in httpd itself? Which seems to indicate that it already should be defined before the attempt by httpd to load the shared object.

Code:
[root@webdav ~]# /usr/local/sbin/httpd -v
Server version: Apache/2.4.51 (FreeBSD)
Server built:   unknown

[root@webdav ~]# nm -D /usr/local/sbin/httpd | grep ap_parse_strict_length
00000000002a9f20 D ap_hack_ap_parse_strict_length
000000000026b9f0 T ap_parse_strict_length
 
Yes, that is correct. As nm() indicates, "D" stands for a global symbol naming initialized data, "T" stands for a global text symbol.
It appears as if the modules somehow doesn't interoperate correctly with the httpd binary. Have you tried a full service stop and reinstalling apache?
 
This occured after an upgrade to FreeBSD-12.2p10 and an upgrade to Apache. The jail was restarted twice during this process. That would indicate that the service likewise was restarted. However, the problem has not reoccurred so we may never know exactly what happend to cause this.
 
Back
Top