8000 Date formulas are not valid in custom error logs · Issue #2884 · haproxy/haproxy · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Date formulas are not valid in custom error logs #2884

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
allegrox opened this issue Feb 28, 2025 · 4 comments
Closed

Date formulas are not valid in custom error logs #2884

allegrox opened this issue Feb 28, 2025 · 4 comments
Labels
status: feedback required The developers are waiting for a reply from the reporter. status: fixed This issue is a now-fixed bug.

Comments

@allegrox
Copy link
allegrox commented Feb 28, 2025

Detailed Description of the Problem

When I use a date formula in a custom error log, the date field is not output in the log. But the output is correct when using date variable (%t). The date formula used here is the one that corresponds to the date variable in the official documentation (https://docs.haproxy.org/3.0/configuration.html#8.2.6).

Expected Behavior

date formulas are not valid in custom error logs

Steps to Reproduce the Behavior

Stable reproduction:

    error-log-format "%ci [%[accept_date(ms),ms_ltime("%d/%b/%Y:%H:%M:%S.%3N")] %ft:%fp %ac/%fc %[fc_err_name]/% ssl_fc_err_str]"

no date in log

    error-log-format "%ci [%t] %ft:%fp %ac/%fc %[fc_err_name]/%[ssl_fc_err_str]"

There's date in the log.

Do you have any idea what may have caused this?

No response

Do you have an idea how to solve the issue?

No response

What is your configuration?

global
    daemon
    maxconn 40960
# enable the following directives for openssl 1.1.1+
#    ssl-default-bind-options no-sslv3 no-tlsv10 no-tlsv11
#    ssl-default-bind-ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256
#    ssl-default-bind-ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256
#    tune.ssl.default-dh-param 2048
    log /dev/log format iso local0 info

defaults
    option dontlognull
    timeout connect   5s
    timeout client   30s
    timeout client-fin 30s
    timeout server   30s
    timeout tunnel   90s

listen http-in
    mode http
    log global
    log-format "%ci [%[accept_date(ms),ms_ltime("%Y-%m-%dT%H:%M:%S.%3N%z")]] %ft:%fp \"%r\" %ST %B %ts %Ta/%Tt %ac/%fc/%bc %si:%sp %hrl"
#    error-log-format "%ci [%[accept_date(ms),ms_ltime("%Y-%m-%dT%H:%M:%S.%3N%z")]] %ft:%fp %ac/%fc %[fc_err_name]/%[ssl_fc_err_str]"
    error-log-format "%ci [%t] %ft:%fp %ac/%fc %[fc_err_name]/%[ssl_fc_err_str]"
    capture request header ztid len 32
    option forwardfor
    bind *:8000-8199 ssl crt /etc/haproxy/cert ssl-min-ver TLSv1.2
    server server1 x.x.x.x

![Image](https://github.com/user-attachments/assets/e72105b9-459f-402e-8344-4e302a6ab419)

 ssl verify none

listen tcp-in
    mode tcp
    log global
    log-format "%ci [%[accept_date(ms),ms_ltime("%Y-%m-%dT%H:%M:%S.%3N%z")]] %ft:%fp %B/%U %ts %Tw/%Tc/%Tt %ac/%fc/%bc"
    error-log-format "%ci [%[accept_date(ms),ms_ltime("%Y-%m-%dT%H:%M:%S.%3N%z")]] %ft:%fp %ts %ac/%fc"
    bind *:8500-8699
    server server1 x.x.x.x

Output of haproxy -vv

HAProxy version 3.0.8-6036c31 2025/01/29 - https://haproxy.org/
Status: long-term supported branch - will stop receiving fixes around Q2 2029.
Known bugs: http://www.haproxy.org/bugs/bugs-3.0.8.html
Running on: Linux 3.10.0-1160.119.1.el7.x86_64 #1 SMP Tue Jun 4 14:43:51 UTC 2024 x86_64
Build options :
  TARGET  = linux-glibc
  CC      = cc
  CFLAGS  = -O2 -g -fwrapv
  OPTIONS = USE_OPENSSL=1 USE_ZLIB=1 USE_CPU_AFFINITY=1 USE_SYSTEMD=1 USE_PCRE=1
  DEBUG   = 

Feature list : -51DEGREES +ACCEPT4 +BACKTRACE -CLOSEFROM +CPU_AFFINITY +CRYPT_H -DEVICEATLAS +DL -ENGINE +EPOLL -EVPORTS +GETADDRINFO -KQUEUE -LIBATOMIC +LIBCRYPT +LINUX_CAP +LINUX_SPLICE +LINUX_TPROXY -LUA -MATH -MEMORY_PROFILING +NETFILTER +NS -OBSOLETE_LINKER +OPENSSL -OPENSSL_AWSLC -OPENSSL_WOLFSSL -OT +PCRE -PCRE2 -PCRE2_JIT -PCRE_JIT +POLL +PRCTL -PROCCTL -PROMEX -PTHREAD_EMULATION -QUIC -QUIC_OPENSSL_COMPAT +RT +SHM_OPEN -SLZ +SSL -STATIC_PCRE -STATIC_PCRE2 +SYSTEMD +TFO +THREAD +THREAD_DUMP +TPROXY -WURFL +ZLIB

Default settings :
  bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with multi-threading support (MAX_TGROUPS=16, MAX_THREADS=256, default=2).
Built with OpenSSL version : OpenSSL 1.0.2k-fips  26 Jan 2017
Running on OpenSSL version : OpenSSL 1.0.2k-fips  26 Jan 2017
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : SSLv3 TLSv1.0 TLSv1.1 TLSv1.2
Built with network namespace support.
Built with zlib version : 1.2.7
Running on zlib version : 1.2.7
Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
Built with PCRE version : 8.32 2012-11-30
Running on PCRE version : 8.32 2012-11-30
PCRE library supports JIT : no (USE_PCRE_JIT not set)
Encrypted password support via crypt(3): yes
Built with gcc compiler version 4.8.5 20150623 (Red Hat 4.8.5-44)

Available polling systems :
      epoll : pref=300,  test result OK
       poll : pref=200,  test result OK
     select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

Available multiplexer protocols :
(protocols marked as <default> cannot be specified using 'proto' keyword)
         h2 : mode=HTTP  side=FE|BE  mux=H2    flags=HTX|HOL_RISK|NO_UPG
  <default> : mode=HTTP  side=FE|BE  mux=H1    flags=HTX
         h1 : mode=HTTP  side=FE|BE  mux=H1    flags=HTX|NO_UPG
       fcgi : mode=HTTP  side=BE     mux=FCGI  flags=HTX|HOL_RISK|NO_UPG
  <default> : mode=TCP   side=FE|BE  mux=PASS  flags=
       none : mode=TCP   side=FE|BE  mux=PASS  flags=NO_UPG

Available services : none

Available filters :
	[BWLIM] bwlim-in
	[BWLIM] bwlim-out
	[CACHE] cache
	[COMP] compression
	[FCGI] fcgi-app
	[SPOE] spoe
	[TRACE] trace

Last Outputs and Backtraces


Additional Information

No response

@allegrox allegrox added status: needs-triage This issue needs to be triaged. type: bug This issue describes a bug. labels Feb 28, 2025
@einval22
Copy link
Contributor
einval22 commented Feb 28, 2025

Hi allegrox !

Thanks for the report I've reproduced the same strange behavior, looking on it.

@einval22 einval22 added the status: reviewed This issue was reviewed. A fix is required. label Feb 28, 2025
@einval22
Copy link
Contributor
einval22 commented Feb 28, 2025

%[accept_date] sample fetch seems to work as expected within the "error-log-format" and it fetches nothing here, because a stream object, associated to the valid ("accepted" in terms of haproxy frontend ) connection does not exist. The stream object is not created in this case, because the incoming connection has failed due to an SSL handshake or invalid PROXY protocol header, as doc says for "error-log-format" here.

To understand more deeply, there are some explanation in the comment from log.c/sess_build_logline() here

You can see the date, when you change this sample fetch to %t, because, when the connection's stream object doesn't exist accept_date is given from the session metadata here and it is just copied here to the building log line string (tmplog variable).

So, when I've reproduced this problem I saw this with %[accept_date] within "error-log-format":

config:
error-log-format ">>>%ci <%[accept_date(ms),ms_ltime("%Y-%m-%dT%H:%M:%S.%3N%z")]> %ft:%fp %ac/%fc %[fc_err_name]"
log:
2025-02-28T15:33:21.228020+00:00 >>>198.18.0.31 - fe_transparent~:1443 1/1 SSL_HANDSHAKE

Instead of date with ms, you have just "-" due to the absence of a stream.

Do you have the same output ?

There is a hint in the doc with an example of quiet full error-log-format
https://docs.haproxy.org/dev/configuration.html#4.3

%tr works well in error-log-format and it also gives milliseconds:

error-log-format ">>>%ci %tr %ft:%fp %ac/%fc %[fc_err_name]"

2025-02-28T15:46:33.889044+00:00 >>>198.18.0.31 28/Feb/2025:15:46:33.888 fe_transparent~:1443 1/1 SSL_HANDSHAKE

@einval22 einval22 added status: feedback required The developers are waiting for a reply from the reporter. status: works as designed This issue stems from a misunderstanding of how HAProxy is supposed to work. and removed type: bug This issue describes a bug. status: needs-triage This issue needs to be triaged. labels Feb 28, 2025
@allegrox
Copy link
Author
allegrox commented Mar 3, 2025

Yes, date replaced by "-" in error-log, That is, in the error log, %t is not derived from the formula?
So my requirement is to record the date in ISO8601 format in the error log, is there any other way to reach it?

8000

@einval22
Copy link
Contributor
einval22 commented Mar 4, 2025

Yes, date replaced by "-" in error-log, That is, in the error log, %t is not derived from the formula? So my requirement is to record the date in ISO8601 format in the error log, is there any other way to reach it?

By initial design we stop to process accept_date sample fetch, if there is no any stream. Date string in such case is formatted in date2str_log() and the format is fixed, "/" is used as delimiters (YY/MM/DD), which is not in the ISO standard.

I've looked to the code again and come up with the patch below. As we always save the accept date, when we build a log line, in the case of error log, the accept time could be also fetched and formatted.

You could apply this patch to the latest dev version and test it, if you are blocked with this:

diff --git a/src/tcp_sample.c b/src/tcp_sample.c
index 109119070..9b6f4adee 100644
--- a/src/tcp_sample.c
+++ b/src/tcp_sample.c
@@ -499,18 +499,25 @@ smp_fetch_accept_date(const struct arg *args, struct sample *smp, const char *kw
        struct strm_logs *logs;
        struct timeval tv;

-       if (!smp->strm)
+       if (smp->strm) {
+               logs = &smp->strm->logs;
+
+               if (kw[0] == 'r') {  /* request_date */
+                       tv_ms_add(&tv, &logs->accept_date, logs->t_idle >= 0 ? logs->t_idle + logs->t_h>
+               } else {             /* accept_date */
+                       tv.tv_sec = logs->accept_date.tv_sec;
+                       tv.tv_usec = logs->accept_date.tv_usec;
+               }
+       } else if (smp->sess) {
+               if (kw[0] == 'r') {  /* request_date */
+                       tv_ms_add(&tv, &smp->sess->accept_date, smp->sess->t_idle >= 0 ? smp->sess->t_i>
+               } else {             /* accept_date */
+                       tv.tv_sec = smp->sess->accept_date.tv_sec;
+                       tv.tv_usec = smp->sess->accept_date.tv_usec;
+               }
+       } else
                return 0;

-       logs = &smp->strm->logs;
-
-       if (kw[0] == 'r') {  /* request_date */
-               tv_ms_add(&tv, &logs->accept_date, logs->t_idle >= 0 ? logs->t_idle + logs->t_handshake>
-       } else {             /* accept_date */
-               tv.tv_sec = logs->accept_date.tv_sec;
-               tv.tv_usec = logs->accept_date.tv_usec;
-       }
-
        smp->data.u.sint = tv.tv_sec;

        /* report in milliseconds */
(END)

Need to pass review and to discuss with collegues, if there is no any side effects by considering the session data in this sample fetch.

@einval22 einval22 added status: fixed This issue is a now-fixed bug. and removed status: works as designed This issue stems from a misunderstanding of how HAProxy is supposed to work. labels Mar 4, 2025
einval22 added a commit to einval22/haproxy that referenced this issue Mar 4, 2025
Sample fetches %[accept_date] and %[request_date] with converters can be used
in error-log-format string. But in the most error cases they fetches nothing,
as error logs are produced on SSL handshake issues or when invalid PROXY
protocol header is used. Stream object is never allocated in such cases and
smp_fetch_accept_date() just simply returns 0.

There is a need to have a custom date format (ISO8601) also in the error logs,
along with normal logs. When sess_build_logline_orig() builds log line it
always copies the accept date to strm_logs structure. When stream is absent,
accept date is copied from the session object.

So, if the steam object wasn't allocated, let's use the session date info in
smp_fetch_accept_date(). This allows then, in sample_process(), to apply to the
fetched date different converters and formats.

This fixes the issue haproxy#2884.
haproxy-mirror pushed a commit that referenced this issue Mar 4, 2025
Sample fetches %[accept_date] and %[request_date] with converters can be used
in error-log-format string. But in the most error cases they fetches nothing,
as error logs are produced on SSL handshake issues or when invalid PROXY
protocol header is used. Stream object is never allocated in such cases and
smp_fetch_accept_date() just simply returns 0.

There is a need to have a custom date format (ISO8601) also in the error logs,
along with normal logs. When sess_build_logline_orig() builds log line it
always copies the accept date to strm_logs structure. When stream is absent,
accept date is copied from the session object.

So, if the steam object wasn't allocated, let's use the session date info in
smp_fetch_accept_date(). This allows then, in sample_process(), to apply to t
7C3C
he
fetched date different converters and formats.

This fixes the issue #2884.
einval22 added a commit to einval22/haproxy that referenced this issue Mar 5, 2025
Sample fetches %[accept_date] and %[request_date] with converters can be used
in error-log-format string. But in the most error cases they fetches nothing,
as error logs are produced on SSL handshake issues or when invalid PROXY
protocol header is used. Stream object is never allocated in such cases and
smp_fetch_accept_date() just simply returns 0.

There is a need to have a custom date format (ISO8601) also in the error logs,
along with normal logs. When sess_build_logline_orig() builds log line it
always copies the accept date to strm_logs structure. When stream is absent,
accept date is copied from the session object.

So, if the steam object wasn't allocated, let's use the session date info in
smp_fetch_accept_date(). This allows then, in sample_process(), to apply to the
fetched date different converters and formats.

This fixes the issue haproxy#2884.
@capflam capflam removed the status: reviewed This issue was reviewed. A fix is required. label Apr 22, 2025
@capflam capflam closed this as completed Apr 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status: feedback required The developers are waiting for a reply from the reporter. status: fixed This issue is a now-fixed bug.
Projects
None yet
Development

No branches or pull requests

3 participants
0