Quantcast
Channel: Serverphorums.com - HAProxy
Viewing all 11674 articles
Browse latest View live
↧

Re: patch to avoid null pointer dereference

October 1, 2018, 9:50 pm
$
0
0
Hi Ilya,

On Tue, Oct 02, 2018 at 09:16:23AM +0500, ???? ??????? wrote:
> this one was found by automation, i.e. coverity.
> I see haproxy was added to coverity in 2015 and builds were not submitted
> since 2015.
>
> shall we send builds, for example monthly ?

It's as you want, the only thing is that I don't want to be the one having
to validate these reports. I accepted to try it by being strongly encouraged
while already convinced of the huge amount of false positives to expect, and
the first report I looked at was filled with 20 false positives among the
first 20 issues, so I stopped looking at it. It already takes a lot of time
to work on real bugs, working on fake ones imagined by a tool which doesn't
know how the code is used is even worse because it's demotivating. So this
is the limit I'm setting.

> (there are several more null pointer dereferences, expect patches from me)

OK. In addition, I really want to have in the commit messages an analysis
of the real world cases where these bugs might trigger, because either they
are possible and we should document it to help people facing these issues,
or they can't happen and we're just sabottaging the code to silence a tool
and this is not acceptable.

Thanks!
Willy
↧
Search

Re: reg-test failures on FreeBSD, how to best adapt/skip some tests?

October 2, 2018, 12:10 am
$
0
0
On 10/01/2018 08:59 PM, PiBa-NL wrote:
> Hi Frederic,
>
> Op 1-10-2018 om 16:09 schreef Frederic Lecaille:
>>> - /connection/b00000.vtc
>>> probably does not 'really' need abns@ sockets, so changing to unix@
>>> would make it testable on more platforms?
>> Correct. I agree I did not think to replace this part specific to Linux.
> Should i send a patch changing that? Or can you fix it easily ;)

Yes, feel free to send a patch for that.

>>
>>> - /log/b00000.vtc
>>> Not exactly sure why this fails/why it was supposed to work.
>>> It either produces a timeout, or the s1 server fails to read the
>>> request which the tcp-healthcheck does not send..
>>>
>>>      ***  s1    0.0 accepted fd 5 127.0.0.1 23986
>>>      **   s1    0.0 === rxreq
>>>      ---- s1    0.0 HTTP rx failed (fd:5 read: Connection reset by peer)
>>
>> Perhaps the syslog traces could give us more information about what is
>> happening here.
> Afaik the 'check' on the server line is failing to send a GET /
> request.. My other mail has a bit more ideas about that..
>>
>>> - /seamless-reload/b00000.vtc
>>> This one specifically mentions testing a abns@ socket functionality.
>>> so changing it to a unix@ socket likely changes the test in such a
>>> way its no longer testing what it was meant for..
>>> What would be the best way to skip this test on FreeBSD?
>>
>> Perhaps we should use the TARGET value to select the VTC files
>> directories which should be selected for the OSes.
>>
>> By default for linux all VTC files in reg-tests directory should be
>> run (found with find command without -L option, so that not to follow
>> the symbolic link).
>>
>> For instance for freebsd OS we would create reg-tests/freebsd directory
>> with symbolic links to the linux reg-tests subdirectories it supports.
> I think creating a list of tests that could be run on FreeBSD will take
> a lot of maintenance and i assume 'most' tests will actually be runnable
> on most OS's. So having short list with exceptions is probably easier to
> maintain. In my opinion every test should be run on every os unless
> there is some good reason not to run it (abns/splicing/stuff..). And if
> possible the reason to exclude a specific test should be described (a
> single line of text could be enough).

I did not mention to create new tests for each OSes. All VTC files would
be created in directories below reg-tests directories. They would be the
union of all VTC files supported by all the OSes supported by HAProxy.
reg-tests directory would contain directories for each OSes with
symbolic links to the directories previously mentioned, if they contain
VTC files supported by this OS.

Something like that:

reg-tests
log
abns

linux
log -> ../log
abns -> ../abns

freebsd
log -> ../log




> The even bigger issue that Willy raised is with compilation options not
> including lua/ssl/gzip/threads/ stuff, and having lots of tests fail on
> those, while they should get skipped if they rely on a feature not
> compiled. I discussed this also a bit with Willy in the other mail
> thread ( https://www.mail-archive.com/haproxy@formilux.org/msg31345.html
> ), but i don't think we have defined the perfect way to do it yet..
>
> So that part of the question still stands ;) .. Whats the best way to
> skip tests that are not applicable?

At this time I do not know how to skip the tests which are not supported
due to disable features at compilation time without running a "haproxy
-vv" command before launching the VTC files.

> Regards,
> PiBa-NL (Pieter)
>
>
↧
↧

HAProxy is not supporting MySQL-8.0 default user authentication plugin (caching_sha2_password)

October 2, 2018, 1:20 am
$
0
0
Hi Team,

HAProxy is not supporting MySQL-8.0 default user authentication plugin
(caching_sha2_password).

HAProxy verison info :

$ haproxy -vv
HA-Proxy version 1.5.18 2016/05/10
Copyright 2000-2016 Willy Tarreau <willy@haproxy.org>

Build options :
TARGET = linux2628
CPU = generic
CC = gcc
CFLAGS = -O2 -g -fno-strict-aliasing
OPTIONS = USE_LINUX_TPROXY=1 USE_ZLIB=1 USE_REGPARM=1 USE_OPENSSL=1
USE_PCRE=1

Default settings :
maxconn = 2000, bufsize = 16384, maxrewrite = 8192, maxpollevents = 200

Encrypted password support via crypt(3): yes
Built with zlib version : 1.2.3
Compression algorithms supported : identity, deflate, gzip
Built with OpenSSL version : OpenSSL 1.0.1e-fips 11 Feb 2013
Running on OpenSSL version : OpenSSL 1.0.1e-fips 11 Feb 2013
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports prefer-server-ciphers : yes
Built with PCRE version : 7.8 2008-09-05
PCRE library supports JIT : no (USE_PCRE_JIT not set)
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT
IP_FREEBIND

Available polling systems :
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use epoll.

$

Error Info :

Sep 27 04:22:55 localhost haproxy[29022]: Server mysql-cluster/node1 is
DOWN, reason: Layer7 wrong status, code: 0, info: "Client does not support
authentication protocol requested by server; consider upgrading MySQL
client", check duration: 0ms. 1 active and 0 backup servers left. 0
sessions active, 0 requeued, 0 remaining in queue.
Sep 27 04:22:56 localhost haproxy[29023]: Server mysql-cluster/node2 is
DOWN, reason: Layer7 wrong status, code: 0, info: "Client does not support
authentication protocol requested by server; consider upgrading MySQL
client", check duration: 0ms. 0 active and 0 backup servers left. 0
sessions active, 0 requeued, 0 remaining in queue.



--
Best Regards,

*Ramesh Sivaraman*
*Senior QA Engineer, Percona*
http://www.percona.com/ http://percona.com/
Phone : +91 8606432991
Skype : rameshvs02
↧

Re: PATCH : mux_h2 h2c pointer deref

October 2, 2018, 8:00 am
$
0
0
OK.
Here you are.

Mildis

> Le 2 oct. 2018 à 04:23, Willy Tarreau <w@1wt.eu> a écrit :
>
> Hi,
>
> On Sun, Sep 23, 2018 at 06:18:37PM +0200, Mildis wrote:
>> Hi,
>>
>> Here is a patch for a null-deref.
>> It checks if h2c exists before working on it.
>>
>
> For these two patches, I'd prefer to have multiple exit labels
> than adding some "if" in the error exit path. It's important that
> the error path is clear, linear and without any ambiguity. For
> example, just add labels likes "fail_no_h2c" and "fail_no_queue"
> pointing to the return. That easily allows to later add intermediary
> branches if some other entries are allocated.
>
> Willy
↧

High CPU Usage followed by segfault error

October 2, 2018, 8:00 am
$
0
0
Hello,

We are currently using haproxy 1.8.3 with single process multithreaded
configuration.
We have 1 process and 10 threads each mapped to a separate core [0-9]. We
are running our haproxy instances on a c4.4xlarge aws ec2 instance. The
only other CPU intensive process running on this server is a log shipper
which is explicity mapped to cpu cores 13 - 16 explicitly using taskset
command. Also we have given 'SCHED_RR' priority 99 for haproxy processes.

OS: Ubuntu 14
Kernel: 4.4.0-134-generic

The issue we are seeing with Haproxy is all of a sudden CPU usage spikes to
100% on cores which haproxy is using & causing latency spikes and high load
on the server. We are seeing the following error messages in system /
kernel logs when this issue happens.

haproxy[92558]: segfault at 8 ip 000055f04b1f5da2 sp 00007ffdab2bdd40 error
6 in haproxy[55f04b10100
0+170000]

Sep 29 12:21:02 marathonlb-int21 kernel: [2223350.996059] sched: RT
throttling activated

We are using marathonlb for auto discovery and reloads are quite frequent
on this server. Last time when this issue happened we had seen haproxy
using 750% of CPU and it went into D state. Also the old process was also
taking cpu.

hard-stop-after was not set in our hap configuration and we were seeing
multiple old pid's running on the server. After the last outage we had with
CPU we set 'hard-stop-after' to 10s and now we are not seeing multiple hap
instances running after reload. I would really appreciate if some one can
explain us why the CPU usage spikes with the above segfault error & what
this error exactly means.

FYI: There was no traffic spike on this hap instance when the issue
happened. We have even seen the same issue in a non-prod hap where no
traffic was coming & system went down due to CPU usage & found the same
segfault error in the logs.

Thanks

Thanks
↧
↧

Few problems seen in haproxy? (threads, connections).

October 2, 2018, 9:00 am
$
0
0
Hi Willy, and community developers,

I am not sure if I am doing something wrong, but wanted to report
some issues that I am seeing. Please let me know if this is a problem.

1. HAProxy system:
Kernel: 4.17.13,
CPU: 48 core E5-2670 v3
Memory: 128GB memory
NIC: Mellanox 40g with IRQ pinning

2. Client, 48 core similar to server. Test command line:
wrk -c 4800 -t 48 -d 30s http://<IP:80>/128

3. HAProxy version: I am testing both 1.8.14 and 1.9-dev3 (git checkout as
of
Oct 2nd).
# haproxy-git -vv
HA-Proxy version 1.9-dev3 2018/09/29
Copyright 2000-2018 Willy Tarreau <willy@haproxy.org>

Build options :
TARGET = linux2628
CPU = generic
CC = gcc
CFLAGS = -O2 -g -fno-strict-aliasing -Wdeclaration-after-statement
-fwrapv -fno-strict-overflow -Wno-unused-label -Wno-sign-compare
-Wno-unused-parameter -Wno-old-style-declaration -Wno-ignored-qualifiers
-Wno-clobbered -Wno-missing-field-initializers -Wtype-limits
OPTIONS = USE_ZLIB=yes USE_OPENSSL=1 USE_PCRE=1

Default settings :
maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with OpenSSL version : OpenSSL 1.0.2g 1 Mar 2016
Running on OpenSSL version : OpenSSL 1.0.2g 1 Mar 2016
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT
IP_FREEBIND
Encrypted password support via crypt(3): yes
Built with multi-threading support.
Built with PCRE version : 8.38 2015-11-23
Running on PCRE version : 8.38 2015-11-23
PCRE library supports JIT : no (USE_PCRE_JIT not set)
Built with zlib version : 1.2.8
Running on zlib version : 1.2.8
Compression algorithms supported : identity("identity"),
deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with network namespace support.

Available polling systems :
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use epoll.

Available multiplexer protocols :
(protocols markes as <default> cannot be specified using 'proto' keyword)
h2 : mode=HTTP side=FE
<default> : mode=TCP|HTTP side=FE|BE

Available filters :
[SPOE] spoe
[COMP] compression
[TRACE] trace

4. HAProxy results for #processes and #threads
# Threads-RPS Procs-RPS
1 20903 19280
2 46400 51045
4 96587 142801
8 172224 254720
12 210451 437488
16 173034 437375
24 79069 519367
32 55607 586367
48 31739 596148

5. Lock stats for 1.9-dev3: Some write locks on average took a lot more time
to acquire, e.g. "POOL" and "TASK_WQ". For 48 threads, I get:
Stats about Lock FD:
# write lock : 143933900
# write unlock: 143933895 (-5)
# wait time for write : 11370.245 msec
# wait time for write/lock: 78.996 nsec
# read lock : 0
# read unlock : 0 (0)
# wait time for read : 0.000 msec
# wait time for read/lock : 0.000 nsec
Stats about Lock TASK_RQ:
# write lock : 2062874
# write unlock: 2062875 (1)
# wait time for write : 7820.234 msec
# wait time for write/lock: 3790.941 nsec
# read lock : 0
# read unlock : 0 (0)
# wait time for read : 0.000 msec
# wait time for read/lock : 0.000 nsec
Stats about Lock TASK_WQ:
# write lock : 2601227
# write unlock: 2601227 (0)
# wait time for write : 5019.811 msec
# wait time for write/lock: 1929.786 nsec
# read lock : 0
# read unlock : 0 (0)
# wait time for read : 0.000 msec
# wait time for read/lock : 0.000 nsec
Stats about Lock POOL:
# write lock : 2823393
# write unlock: 2823393 (0)
# wait time for write : 11984.706 msec
# wait time for write/lock: 4244.788 nsec
# read lock : 0
# read unlock : 0 (0)
# wait time for read : 0.000 msec
# wait time for read/lock : 0.000 nsec
Stats about Lock LISTENER:
# write lock : 184
# write unlock: 184 (0)
# wait time for write : 0.011 msec
# wait time for write/lock: 60.554 nsec
# read lock : 0
# read unlock : 0 (0)
# wait time for read : 0.000 msec
# wait time for read/lock : 0.000 nsec
Stats about Lock PROXY:
# write lock : 291557
# write unlock: 291557 (0)
# wait time for write : 109.694 msec
# wait time for write/lock: 376.235 nsec
# read lock : 0
# read unlock : 0 (0)
# wait time for read : 0.000 msec
# wait time for read/lock : 0.000 nsec
Stats about Lock SERVER:
# write lock : 1188511
# write unlock: 1188511 (0)
# wait time for write : 854.171 msec
# wait time for write/lock: 718.690 nsec
# read lock : 0
# read unlock : 0 (0)
# wait time for read : 0.000 msec
# wait time for read/lock : 0.000 nsec
Stats about Lock LBPRM:
# write lock : 1184709
# write unlock: 1184709 (0)
# wait time for write : 778.947 msec
# wait time for write/lock: 657.501 nsec
# read lock : 0
# read unlock : 0 (0)
# wait time for read : 0.000 msec
# wait time for read/lock : 0.000 nsec
Stats about Lock BUF_WQ:
# write lock : 669247
# write unlock: 669247 (0)
# wait time for write : 252.265 msec
# wait time for write/lock: 376.939 nsec
# read lock : 0
# read unlock : 0 (0)
# wait time for read : 0.000 msec
# wait time for read/lock : 0.000 nsec
Stats about Lock STRMS:
# write lock : 9335
# write unlock: 9335 (0)
# wait time for write : 0.910 msec
# wait time for write/lock: 97.492 nsec
# read lock : 0
# read unlock : 0 (0)
# wait time for read : 0.000 msec
# wait time for read/lock : 0.000 nsec
Stats about Lock VARS:
# write lock : 901947
# write unlock: 901947 (0)
# wait time for write : 299.224 msec
# wait time for write/lock: 331.753 nsec
# read lock : 0
# read unlock : 0 (0)
# wait time for read : 0.000 msec
# wait time for read/lock : 0.000 nsec

6. CPU utilization after test for processes/threads: haproxy-1.9-dev3 runs
at 4800% (48 cpus) for 30 seconds after the test is done. For 1.8.14,
this behavior was not seen. Ran the following command for both:
"ss -tnp | awk '{print $1}' | sort | uniq -c | sort -n"
1.8.14 during test:
451 SYN-SENT
9166 ESTAB
1.8.14 after test:
2 ESTAB

1.9-dev3 during test:
109 SYN-SENT
9400 ESTAB
1.9-dev3 after test:
2185 CLOSE-WAIT
2187 ESTAB
All connections that were in CLOSE-WAIT were from the client, while all
connections in ESTAB state were to the server. This lasted for 30
seconds.
On the client system, all sockets were in FIN-WAIT-2 state:
2186 FIN-WAIT-2
This (2185/2186) seems to imply that client closed the connection but
haproxy did not close the socket for 30 seconds. This also results in
high CPU utilization on haproxy for some reason (100% for each process
for 30 seconds), which is also unexpected as the remote side has closed
the
socket.

7. Configuration file for process mode:
global
daemon
maxconn 26000
nbproc 48
stats socket /var/run/ha-1-admin.sock mode 600 level admin process 1
# (and so on for 48 processes).

defaults
option http-keep-alive
balance leastconn
retries 2
option redispatch
maxconn 25000
option splice-response
option tcp-smart-accept
option tcp-smart-connect
option splice-auto
timeout connect 5000ms
timeout client 30000ms
timeout server 30000ms
timeout client-fin 30000ms
timeout http-request 10000ms
timeout http-keep-alive 75000ms
timeout queue 10000ms
timeout tarpit 15000ms

frontend fk-fe-upgrade-80
mode http
default_backend fk-be-upgrade
bind <VIP>:80 process 1
# (and so on for 48 processes).

backend fk-be-upgrade
mode http
default-server maxconn 2000 slowstart
# 58 server lines follow, e.g.: "server <name> <ip:80>"

8. Configuration file for thread mode:
global
daemon
maxconn 26000
stats socket /var/run/ha-1-admin.sock mode 600 level admin
nbproc 1
nbthread 48
# cpu-map auto:1/1-48 0-39

defaults
option http-keep-alive
balance leastconn
retries 2
option redispatch
maxconn 25000
option splice-response
option tcp-smart-accept
option tcp-smart-connect
option splice-auto
timeout connect 5000ms
timeout client 30000ms
timeout server 30000ms
timeout client-fin 30000ms
timeout http-request 10000ms
timeout http-keep-alive 75000ms
timeout queue 10000ms
timeout tarpit 15000ms

frontend fk-fe-upgrade-80
mode http
bind <VIP>:80 process 1/1-48
default_backend fk-be-upgrade

backend fk-be-upgrade
mode http
default-server maxconn 2000 slowstart
# 58 server lines follow, e.g.: "server <name> <ip:80>"

I had also captured 'perf' output for the system for thread vs processes,
can send it later if required.

Thanks,
- Krishna
↧

Re: High CPU Usage followed by segfault error

October 2, 2018, 9:50 am
$
0
0
Hi,

On Tue, Oct 02, 2018 at 08:26:12PM +0530, Soji Antony wrote:
> Hello,
>
> We are currently using haproxy 1.8.3 with single process multithreaded
> configuration.
> We have 1 process and 10 threads each mapped to a separate core [0-9]. We
> are running our haproxy instances on a c4.4xlarge aws ec2 instance. The
> only other CPU intensive process running on this server is a log shipper
> which is explicity mapped to cpu cores 13 - 16 explicitly using taskset
> command. Also we have given 'SCHED_RR' priority 99 for haproxy processes.
>
> OS: Ubuntu 14
> Kernel: 4.4.0-134-generic
>
> The issue we are seeing with Haproxy is all of a sudden CPU usage spikes to
> 100% on cores which haproxy is using & causing latency spikes and high load
> on the server. We are seeing the following error messages in system /
> kernel logs when this issue happens.
>
> haproxy[92558]: segfault at 8 ip 000055f04b1f5da2 sp 00007ffdab2bdd40 error
> 6 in haproxy[55f04b10100
> 0+170000]
>
> Sep 29 12:21:02 marathonlb-int21 kernel: [2223350.996059] sched: RT
> throttling activated
>
> We are using marathonlb for auto discovery and reloads are quite frequent
> on this server. Last time when this issue happened we had seen haproxy
> using 750% of CPU and it went into D state. Also the old process was also
> taking cpu.
>
> hard-stop-after was not set in our hap configuration and we were seeing
> multiple old pid's running on the server. After the last outage we had with
> CPU we set 'hard-stop-after' to 10s and now we are not seeing multiple hap
> instances running after reload. I would really appreciate if some one can
> explain us why the CPU usage spikes with the above segfault error & what
> this error exactly means.
>
> FYI: There was no traffic spike on this hap instance when the issue
> happened. We have even seen the same issue in a non-prod hap where no
> traffic was coming & system went down due to CPU usage & found the same
> segfault error in the logs.
>

A good first step would probably to upgrade to the latest version if possible.
1.8.3 is quite old, and a bunch of bugs have been fixed since then,
especially when using multithreading.

Regards,

Olivier
↧

Redirect to HTTPS

October 2, 2018, 11:40 am
$
0
0
I would like to redirect everything from HTTP to HTTPS except a specific URL. Here is what I have but it doesn’t seem to be working.

redirect scheme https if !{ ssl_fc } OR !{ hdr(Host) -m -I www.blah.com }

Thanks,
↧
Search

Re: Redirect to HTTPS

October 2, 2018, 11:50 am
$
0
0
On Tue, 2 Oct 2018 at 20:34, Dustin Schuemann <dschuemann@gmail.com> wrote:
>
> I would like to redirect everything from HTTP to HTTPS except a specific URL.

You mean Host header? Because that's what you configured.


> redirect scheme https if !{ ssl_fc } OR !{ hdr(Host) -m -I www.blah.com }

The logic is flawed. If you don't want to redirect when the host is
www.blah.com, then you need to AND this, not OR. Also the ACL
expression is wrong.

This would be it:
redirect scheme https if !{ ssl_fc } !{ hdr(host) -i www.blah.com }



Lukas
↧
↧

[PATCH] MINOR: generate-certificates for BoringSSL

October 3, 2018, 4:50 am
$
0
0
Hi,

For generate-certificates, X509V3_EXT_conf is used but it's an (very) old API
call: X509V3_EXT_nconf must be preferred. Openssl compatibility is ok
because it's inside #ifdef SSL_CTRL_SET_TLSEXT_HOSTNAME, introduce 5
years after X509V3_EXT_nconf.
(BoringSSL only have X509V3_EXT_nconf)

Christopher, if you have time to check this little patch :)

++
Manu
↧

RE: faster than load-server-state-from-file?

October 3, 2018, 5:10 am
$
0
0
Hi Willy,

> Not really. Maybe we should see how the state file parser works, because
> multiple seconds to parse only 30K lines seems extremely long.

I would even say multiple minutes :)

> I'm just thinking about a few things. Probably that among these 30K servers,
> most of them are in fact tracking other ones ? In this case it could make
> sense to have an option to only dump servers which are not tracking
> others, as for a reload it can make quite some sense. Is this the case
> for you ?

What do you mean by "tracking other ones"?

What I can tell is that, for historical reasons, we named all server the same way for each backends (ie. srvN) in the configuration template, and are using "server templates" to add MAINT servers in the pool so that they can be added at runtime later.

This naming thing can be changed now, but I don't know this issue could be related or not.

What we're doing basically when getting a new event:
* if it requires to delete / update / add server(s) in one or multiple pools we only use the runtime API and try to reuse free slots.
* if a backend/frontend has to be created / updated / deleted OR if the free slots for a given backend is full we reload using a configuration template.
* in Jinja2 this template looks like (simplified):

backend be_foo
<options>
{%- for server in servers %}
server srv{{loop.index0}} {{server.address}}:{{server.port}} weight {{server.weight}}{%- if server.tls %} ssl{%- endif %} check port 8500
{%- endfor %}
# Create 25 free slots, servers are numbered from N to N+25
server-template srv {{ servers|length }}-{{ servers|length + 25 }} 0.0.0..0:0 check disabled

Doing this I noticed that we have a lot of 'bad reconciliations' triggering warning logs, such as:

[WARNING] can't find server 'srv28' with id '29' in backend with id '9' or name 'be_test'
[WARNING] backend name mismatch: from server state file: 'be_foo', from running config 'be_bar'

I don't know if these inconsistencies (that clearly have to be fixed) can cause additional delays.

Thanks,

Pierre
↧

haproxy start problem

October 3, 2018, 8:00 am
$
0
0
I'm new to this list and I subscribed hoping to get help on an
installation problem. OS is ubuntu 18.04

and haproxy ist the latest stable version 1.8.

systemctl restart haproxy and a subsequent

systemctl status haproxy.service

gives:


● haproxy.service - HAProxy Load Balancer
   Loaded: loaded (/lib/systemd/system/haproxy.service; disabled;
vendor preset: enabled)
   Active: failed (Result: exit-code) since Wed 2018-10-03 16:43:15
CEST; 14s ago
     Docs: man:haproxy(1)
           file:/usr/share/doc/haproxy/configuration.txt.gz
  Process: 7535 ExecStart=/usr/sbin/haproxy -Ws -f $CONFIG -p $PIDFILE
$EXTRAOPTS (code=exited, status=1/FAILURE)
  Process: 7534 ExecStartPre=/usr/sbin/haproxy -f $CONFIG -c -q
$EXTRAOPTS (code=exited, status=0/SUCCESS)
 Main PID: 7535 (code=exited, status=1/FAILURE)

Oct 03 16:43:14 myserver.org systemd[1]: haproxy.service: Main process
exited, code=exited, status=1/FAILURE
Oct 03 16:43:14 myserver.org systemd[1]: haproxy.service: Failed with
result 'exit-code'.
Oct 03 16:43:14 myserver.org systemd[1]: Failed to start HAProxy Load
Balancer.
Oct 03 16:43:15 myserver.org systemd[1]: haproxy.service: Service
hold-off time over, scheduling restart.
Oct 03 16:43:15 myserver.org systemd[1]: haproxy.service: Scheduled
restart job, restart counter is at 5.
Oct 03 16:43:15 myserver.org systemd[1]: Stopped HAProxy Load Balancer.
Oct 03 16:43:15 myserver.org systemd[1]: haproxy.service: Start request
repeated too quickly.
Oct 03 16:43:15 myserver.org systemd[1]: haproxy.service: Failed with
result 'exit-code'.
Oct 03 16:43:15 myserver.org systemd[1]: Failed to start HAProxy Load
Balancer.


/etc/haproxy.cfg:

global
        log /dev/log    local0
        log /dev/log    local1 notice
        chroot /var/lib/haproxy
        stats socket /run/haproxy/admin.sock mode 660 level admin
        stats timeout 30s
        user haproxy
        group haproxy
        daemon

        # Default SSL material locations
        ca-base /etc/ssl/certs
        crt-base /etc/ssl/private

        # Default ciphers to use on SSL-enabled listening sockets.
        # For more information, see ciphers(1SSL). This list is from:
        # https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/
        ssl-default-bind-ciphers
ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS
        ssl-default-bind-options no-sslv3

defaults
        log     global
        mode    http
        option  httplog
        option  dontlognull
        timeout connect 5000
        timeout client  50000
        timeout server  50000
        errorfile 400 /etc/haproxy/errors/400.http
        errorfile 403 /etc/haproxy/errors/403.http
        errorfile 408 /etc/haproxy/errors/408.http
        errorfile 500 /etc/haproxy/errors/500.http
        errorfile 502 /etc/haproxy/errors/502.http
        errorfile 503 /etc/haproxy/errors/503.http
        errorfile 504 /etc/haproxy/errors/504.http
    log global
    mode http
    compression algo gzip
    compression type text/html text/css text/plain text/vcard
text/vnd.rim.location.xloc text/vtt text/x-component
text/x-cross-domain-policy application/atom+xml application/javascript
application/x-javascript application/json application/ld+json
application/manifest+json application/rss+xml application/vnd.geo+json
application/vnd.ms-fontobject application/x-font-ttf
application/x-web-app-manifest+json application/xhtml+xml
application/xml font/opentype image/bmp image/svg+xml image/x-icon
text/cache-manifest
    balance roundrobin
    option dontlog-normal
    option dontlognull
    option httpclose
    option forwardfor

frontend http-in
    bind *:80
    acl is_static       path_beg /export/ /opencms/ /resources/
/javadoc/ /VAADIN/ /workplace /opencms-login/
    acl is_website      hdr_beg(host) -i website
    use_backend website-static if is_website is_static server
cms.myserver.org 127.0.0.1:8080
    use_backend website if is_website

backend website-static
    server cms.myserver.org 127.0.0.1:8080

backend website
    reqirep ^([^\ :]*)\ /(.*) \1\ /opencms/\2
    server www.myserver.org 127.0.0.1:8080


------------------------------





Tomcat is listening on localhost:8080

netstat -an (excerpt):

tcp        0      0 127.0.0.1:8080 0.0.0.0:*               LISTEN


any clues what might be wrong?


--

Christoph
↧

Re: haproxy start problem

October 3, 2018, 8:20 am
$
0
0
Am 03.10.2018 um 17:03 schrieb Hossein Aghaie:
> haproxy -c -f /etc/haproxy/haproxy.cfg

OK, it says


Configuration file is valid
↧
↧

Re: haproxy start problem

October 3, 2018, 9:10 am
$
0
0
Am 03.10.2018 um 17:08 schrieb Christoph P.U. Kukulies:
> Am 03.10.2018 um 17:03 schrieb Hossein Aghaie:
>> haproxy -c -f /etc/haproxy/haproxy.cfg
>
> OK, it says
>
>
> Configuration file is valid
>
>
>
Solved it. Apache2 was still running. Was a bit tricky to remove apache2
from ubuntu 18.04 since it seems to be wired into the system by default.


apt remove apache2


solved it although I wish I had a less rigorous means.


--

Christoph
↧

Re: reg-test failures on FreeBSD, how to best adapt/skip some tests?

October 3, 2018, 1:00 pm
$
0
0
Hi Frederic,

> Op 2-10-2018 om 8:58 schreef Frederic Lecaille:
>> At this time I do not know how to skip the tests which are not supported
>> due to disable features at compilation time without running a
>> "haproxy -vv" command before launching the VTC files.

Made a little script (attached)..

It would allow for the .vtc files to contain what 'requirements' they
have, by including lines like these:

#EXCLUDE_TARGET=freebsd, abns sockets are not available on freebsd
#REQUIRE_OPTION=OPENSSL, this test needs ssl compiled in.

Which could then generate output like listed below. It searches for all
testcases, validates their requirements, and then puts them in a list.
After which it calls varnishtest with quiet and simultaneous job
parameters to allow for a short turnaround time..

What do you think ? Could this work for current 'requirements'? Or is
something different preferred? I though about running each test directly
from the script, and while that allows per test OK/Fail response logging
for and a nicer overall summery, it takes more time to run. So i went
for automatically creating a list of tests and letting varnishtest
itself handle the running of multiple tests at once.

Thoughts appreciated :).

Regards,

PiBa-NL (Pieter)

root@freebsd11:/usr/ports/net/haproxy-devel # ./run-regtests.sh
########################## Preparing to run tests ##########################
HA-Proxy version 1.8.14-52e4d43 2018/09/20
Copyright 2000-2018 Willy Tarreau <willy@haproxy.org>

########################## Gathering tests to run ##########################
Skip ./haproxy_test_OK_20180831/lua/b00002.vtc because option OPENSSL
not found
  REASON:  this test needs ssl compiled in.
Add test: ./haproxy_test_OK_20180831/lua/h00001.vtc
Add test: ./haproxy_test_OK_20180831/lua/b00000.vtc
Add test: ./haproxy_test_OK_20180831/lua/b00001.vtc
Add test: ./haproxy_test_OK_20180831/loadtest/b00000-loadtest.vtc
Add test: ./haproxy_test_OK_20180831/log/b00000.vtc
Skip ./haproxy_test_OK_20180831/reload/b00001-server-state-file.vtc
because: TARGET = freebsd
  REASON:  abns sockets are not available on freebsd
Skip ./haproxy_test_OK_20180831/ssl/b00000.vtc because option OPENSSL
not found
  REASON:  this test needs ssl compiled in.
Add test: ./haproxy_test_OK_20180831/stick-table/b00001.vtc
Add test: ./haproxy_test_OK_20180831/stick-table/b00000.vtc
Add test: ./test/b00000-loadtest.vtc
Skip ./test/b00002.vtc because option OPENSSL not found
  REASON:  this test needs ssl compiled in.
Add test: ./test/b00003-cpu.vtc
Add test: ./work/haproxy-1.8-52e4d43/reg-tests/server/b00000.vtc
Add test: ./work/haproxy-1.8-52e4d43/reg-tests/stick-table/h00000.vtc
Skip ./work/haproxy-1.8-52e4d43/reg-tests/lua/b00002.vtc because option
OPENSSL not found
  REASON:  this test needs ssl compiled in.
Add test: ./work/haproxy-1.8-52e4d43/reg-tests/lua/h00001.vtc
########################## Starting varnishtest ##########################
#    top  TEST ./test/b00003-cpu.vtc FAILED (5.281) exit=2
1 tests failed, 0 tests skipped, 11 tests passed
root@freebsd11:/usr/ports/net/haproxy-devel #

#!/usr/bin/env sh
echo "########################## Preparing to run tests ##########################"
haproxy -v

#varnishtest -j 16 -k -t 20 ./work/haproxy-*/reg-tests/*/*.vtc > ./mytest-result.log 2>&1
#varnishtest -j 16 -k -t 20 ./haproxy_test_OK_20180831/*/*.vtc >> ./mytest-result.log 2>&1
#cat ./mytest-result.log
#echo "" >> ./mytest-result.log
#haproxy -vv >> ./mytest-result.log

TARGET=$(haproxy -vv | grep "TARGET = " | sed 's/.*= //')
OPTIONS=$(haproxy -vv | grep "OPTIONS = " | sed 's/.*= //')

echo "TARGET : $TARGET"
echo "OPTIONS : $OPTIONS"

haproxy -v > testresult_summary.log
haproxy -v > testresult_detailed.log
echo -n > testlist.lst

echo "########################## Gathering tests to run ##########################"
for i in $(find ./ -name "*.vtc");
do
optionismissing=
for required in "$(cat $i | grep REQUIRE_OPTION)";
do
if [ -z "$required" ]
then
continue
fi
requiredoption=$(echo "$required" | sed 's/.*=//' | sed 's/,.*//')
#echo " Test requires: $requiredoption"
if [ -z "$( echo "$OPTIONS" | grep "USE_$requiredoption=1" )" ]
then
echo -n "Skip $i because option $requiredoption not found! REASON: "
echo "$required" | sed 's/.*,//'
optionismissing=1
fi
done

if [ $optionismissing ]
then
continue
fi

testtarget=$(cat $i | grep "#EXCLUDE_TARGET")
#echo "testtarget : $testtarget"
if [ "$( echo "$testtarget" | grep "#EXCLUDE_TARGET=$TARGET," )" ]
then
echo "Skip $i because: TARGET = $TARGET"
echo -n " REASON: "
echo "$testtarget" | sed 's/.*,//'
continue
fi

echo "Add test: $i"
echo "$i" >> testlist.lst
done
echo "########################## Starting varnishtest ##########################"
varnishtest -q -l -j 16 -k $(cat testlist.lst)
↧
Search

[PATCH] REGTEST/MINOR: compatibility: use unix@ instead of abns@

October 3, 2018, 3:10 pm
$
0
0
Hi Frederic, Willy,

Attached a patch that will change /reg-tests/connection/b00000.vtc to
use unix@ sockets so it is compatible with FreeBSD and possibly other OS's.

As discussed in the other thread
https://www.mail-archive.com/haproxy@formilux.org/msg31370.html.

Regards,
PiBa-NL (Pieter)

From 8c5ff12b4603e3525445d6f708f6239974003df4 Mon Sep 17 00:00:00 2001
From: PiBa-NL <PiBa.NL.dev@gmail.com>
Date: Wed, 3 Oct 2018 23:54:49 +0200
Subject: [PATCH] REGTEST/MINOR: compatibility: use unix@ instead of abns@
sockets

Changes the /reg-tests/connection/b00000.vtc test to use unix@ instead of abns@ sockets.
This to allow the test to complete on other operating systems like FreeBSD that do not have 'namespaces'.
---
reg-tests/connection/b00000.vtc | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/reg-tests/connection/b00000.vtc b/reg-tests/connection/b00000.vtc
index 3a873848..cbb8a7b0 100644
--- a/reg-tests/connection/b00000.vtc
+++ b/reg-tests/connection/b00000.vtc
@@ -36,14 +36,14 @@ haproxy h1 -conf {

listen http
bind-process 1
- bind abns@http accept-proxy name ssl-offload-http
+ bind unix@${testdir}/http.socket accept-proxy name ssl-offload-http
option forwardfor

listen ssl-offload-http
option httplog
bind-process 2-4
bind "fd@${ssl}" ssl crt ${testdir}/common.pem ssl no-sslv3 alpn h2,http/1.1
- server http abns@http send-proxy
+ server http unix@${testdir}/http.socket send-proxy
} -start


--
2.18.0.windows.1
↧

Redirecting one https site to another

October 3, 2018, 3:10 pm
$
0
0
Hi,

I'm not sure if this is possible as haproxy isn't terminating SSL in this instance, but I'd like to redirect https://urlone.co.uk to https://www.urlone.co.uk

I have urlone.co.uk pointed to 185.90.33.47 via a DNS A record

bind 181.70.33.47:80
redirect location https://www.urlone.co.uk:443

bind 181.70.33.47:443
redirect location https://www.urlone.co.uk:443


www.urlone.co.ukhttp://www.urlone.co.uk is pointed to 185.90.33.48 via a DNS A record and I have a config like this:

frontend in-redirect-ssl-www.urlone.co.uk
mode http
bind 181.70.33.48:80
redirect scheme https if !{ ssl_fc }

frontend in-www.urlone.co.uk
mode tcp
bind 181.70.33.48:443
default_backend www.urlone.co.uk

backend www.urlone.co.uk
mode tcp
balance roundrobin
stick-table type ip size 200k expire 30m
stick on src
server prod-web-01 192.168.33.211:443 check port 443
server prod-web-02 192.168.33.212:443 check port 443
server Sorry_Server 192.168.33.200:80 check backup


When I hit urlone.co.uk on http I get redirected to https://www.urlone.co.uk. All good. However when I hit urlone.co.uk on https it fails with 'This site can't provide a secure connection' (jn Chrome, message is probably different in other browsers)

Is what I am trying to achieve possible? Grateful for any suggestions.

Thanks,

Mark

BI WORLDWIDE Limited | Registered in England No 01445905 | Registered address 1 Vantage Court, Newport Pagnell, Bucks, MK16 9EZ | +44 (0) 1908 214 700

This e-mail message is being sent solely for use by the intended recipient(s) and may contain confidential information. Any unauthorised review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by phone or reply by e-mail, delete the original message and destroy all copies. Thank you

Please consider the environment before printing this email
↧
↧

Re: Redirecting one https site to another

October 3, 2018, 3:20 pm
$
0
0
Hi Mark,


On Thu, 4 Oct 2018 at 00:03, Mark Holmes <Mark.Holmes@eu.biworldwide.com> wrote:
>
> Hi,
>
>
>
> I’m not sure if this is possible as haproxy isn’t terminating SSL in this instance,
> but I’d like to redirect https://urlone.co.uk to https://www.urlone.co.uk
> [...]
> Is what I am trying to achieve possible? Grateful for any suggestions.

No. A redirect happens at HTTP level, with a HTTP response like 301 or
302. If SSL is just passing through (port 443 in tcp mode), everything
is encrypted. You cannot read HTTP request or insert HTTP responses in
this case.

If you want to redirect, you need access at HTTP level, and to have
that access, you need to terminate SSL at haproxy.



Regards,
Lukas
↧

RE: [EXTERNAL] Re: Redirecting one https site to another

October 3, 2018, 3:30 pm
$
0
0
Lukas - many thanks - that confirms what I thought.

Mark


-----Original Message-----
From: lists@ltri.eu <lists@ltri.eu>
Sent: 03 October 2018 23:15
To: Mark Holmes <Mark.Holmes@eu.biworldwide.com>
Cc: haproxy <haproxy@formilux.org>
Subject: [EXTERNAL] Re: Redirecting one https site to another

Hi Mark,


On Thu, 4 Oct 2018 at 00:03, Mark Holmes <Mark.Holmes@eu.biworldwide.com> wrote:
>
> Hi,
>
>
>
> I’m not sure if this is possible as haproxy isn’t terminating SSL in
> this instance, but I’d like to redirect
> https://urlone.co.uk
> o.uk to
> https://www.urlone.co.uk
> o.uk
> [...]
> Is what I am trying to achieve possible? Grateful for any suggestions.

No. A redirect happens at HTTP level, with a HTTP response like 301 or 302. If SSL is just passing through (port 443 in tcp mode), everything is encrypted. You cannot read HTTP request or insert HTTP responses in this case.

If you want to redirect, you need access at HTTP level, and to have that access, you need to terminate SSL at haproxy.



Regards,
Lukas

BI WORLDWIDE Limited | Registered in England No 01445905 | Registered address 1 Vantage Court, Newport Pagnell, Bucks, MK16 9EZ | +44 (0) 1908 214 700

This e-mail message is being sent solely for use by the intended recipient(s) and may contain confidential information. Any unauthorised review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by phone or reply by e-mail, delete the original message and destroy all copies. Thank you

Please consider the environment before printing this email
↧

Re: [PATCH] REGTEST/MINOR: compatibility: use unix@ instead of abns@

October 3, 2018, 7:30 pm
$
0
0
Hi Pieter,

On Thu, Oct 04, 2018 at 12:02:11AM +0200, PiBa-NL wrote:
> Hi Frederic, Willy,
>
> Attached a patch that will change /reg-tests/connection/b00000.vtc to use
> unix@ sockets so it is compatible with FreeBSD and possibly other OS's.
(...)

applied, thank you!
Willy
↧
Viewing all 11674 articles
Browse latest View live
Search