I think the upstream fix is buggy (remotely triggered memory leak, functional regression); I sent this to the netbsd bug tracker; I'll watch what they have to say.

While reviewing the fix for #51682, I found what I believe is a
newly introduced regression / memory leak / DoS.

Bug scenario: the fragments arrive out of order; iph1->frag_chain
already contains some fragments; the newly received fragment has
the lowest frag_num so it should be inserted in the beginning of
the list:

> RCS file: /cvsroot/src/crypto/dist/ipsec-tools/src/racoon/isakmp_frag.c,v
> retrieving revision 1.5
> diff -u -u -r1.5 isakmp_frag.c
> --- isakmp_frag.c	22 Apr 2009 11:24:20 -0000	1.5
> +++ isakmp_frag.c	24 Jan 2017 02:29:15 -0000
> @@ -173,6 +173,38 @@
>  	return ntohl(hp[MD5_DIGEST_LENGTH / sizeof(*hp)]);
>  }
>
> +static int 
> +isakmp_frag_insert(struct ph1handle *iph1, struct isakmp_frag_item *item)
> +{
> +	struct isakmp_frag_item *pitem = NULL;
> +	struct isakmp_frag_item *citem = iph1->frag_chain;
> +

item is not the first to arrive, iph1->frag_chain != NULL,
so the fragment is not inserted here:

> +	if (iph1->frag_chain == NULL) {
> +		iph1->frag_chain = item;
> +		return 0;
> +	}
> +
> +	do {
> +		if (citem->frag_num == item->frag_num)
> +			return -1;
> +

item is lowest-numbered fragment so we want to insert it now ...

> +		if (citem->frag_num > item->frag_num) {

... but pitem is still NULL, we break doing nothing

> +			if (pitem)
> +				pitem->frag_next = item;
> +			item->frag_next = citem;
> +			break;
> +		}
> +
> +		pitem = citem;
> +		citem = citem->frag_next;
> +	} while (citem != NULL);
> +
> +	/* we reached the end of the list, insert */
> +	if (citem == NULL)
> +	      pitem->frag_next = item;
> +	return 0;

We return 0.
The newly received fragment is not inserted in the list -> it's
lost (a regression) and never freed (possible DoS).

I think the correct fix would either need more special cases or
something simpler like: (untested)

static int 
isakmp_frag_insert(struct ph1handle *iph1, struct isakmp_frag_item *item)
{
	struct isakmp_frag_item **pitem = &iph1->frag_chain
	struct isakmp_frag_item *citem = iph1->frag_chain;

	while (citem != NULL) {
		if (citem->frag_num == item->frag_num)
			return -1;

		if (citem->frag_num > item->frag_num) {
			item->frag_next = citem;
			break;
		}

		pitem = &citem->frag_next;
		citem = citem->frag_next;
	}

	(*pitem)->frag_next = item;
	return 0;
}

This is an autogenerated message for OBS integration:
This bug (1047443) was mentioned in
https://build.opensuse.org/request/show/570012 42.3 / ipsec-tools

An update workflow for this issue was started.
This issue was rated as moderate.
Please submit fixed packages until 2018-02-13.
When done, reassign the bug to security-team@suse.de.
https://swamp.suse.de/webswamp/wf/63959

all versions submitted to BS, reassigning to security-team

SUSE-SU-2018:0423-1: An update that fixes one vulnerability is now available.

Category: security (moderate)
Bug References: 1047443
CVE References: CVE-2016-10396
Sources used:
SUSE Linux Enterprise Server 11-SP4 (src):    ipsec-tools-0.7.3-1.38.3.1
SUSE Linux Enterprise Debuginfo 11-SP4 (src):    ipsec-tools-0.7.3-1.38.3.1

SUSE-SU-2018:0424-1: An update that fixes one vulnerability is now available.

Category: security (moderate)
Bug References: 1047443
CVE References: CVE-2016-10396
Sources used:
SUSE Linux Enterprise Server for Raspberry Pi 12-SP2 (src):    ipsec-tools-0.8.0-19.3.1
SUSE Linux Enterprise Server 12-SP3 (src):    ipsec-tools-0.8.0-19.3.1
SUSE Linux Enterprise Server 12-SP2 (src):    ipsec-tools-0.8.0-19.3.1

released

openSUSE-SU-2018:0434-1: An update that fixes one vulnerability is now available.

Category: security (moderate)
Bug References: 1047443
CVE References: CVE-2016-10396
Sources used:
openSUSE Leap 42.3 (src):    ipsec-tools-0.8.0-14.3.1