TCP/IP listeners

[Thomas Simonds]

Is there any advantage to having multiple TCPIPSRVICE definitions with the same IP address and port # when servicing a high volume of concurrent transactions? I have a SOAP application that was experiencing frequent connection resets and was unable to find the reason. When I added a second listener the problem seemed to have gone away. I wondered if anyone else had experience with this.

Tom Simonds
Cogsdale Corporation
703-727-2678

[i***@winwholesale.com]

Post by Thomas Simonds
Is there any advantage to having multiple TCPIPSRVICE definitions
with the same IP address and port # when servicing a high volume of
concurrent transactions? I have a SOAP application that was
experiencing frequent connection resets and was unable to find the
reason. When I added a second listener the problem seemed to have
gone away.

My understanding is that you shouldn't (or, legitimately, can't?)
have more than one listener on the same IP address for the same port. I'd
be interested in any correction to my understanding. As for high volumes,
I've also been told that if the listener is written correctly, you'd have
to consistently have more than 64 simultaneous connections attempted
before you'd have trouble. Of course, in CICS, you'd be in trouble,
anyway, trying to handle that many spawned child processes at the same
time.

Sincerely,

Dave Clark

WinWholesale Group Services
3110 Kettering Boulevard
Dayton, Ohio 45439 USA
[phone]


This email message and any attachments is for use only by the named
addressee(s) and may contain confidential, privileged and/or proprietary
information. If you have received this message in error, please
immediately notify the sender and delete and destroy the message and all
copies. All unauthorized direct or indirect use or disclosure of this
message is strictly prohibited. No right to confidentiality or privilege
is waived or lost by any error in transmission.

[Thompson, Glenn]

I agree with you Dave. The IP Address + port is the lowest level of
addressing.



I'm wondering if the initial listener had stopped working and defining
the second listener allowed the application to function normally.





From: owner-vse-***@Lehigh.EDU [mailto:owner-vse-***@Lehigh.EDU] On Behalf
Of ***@winwholesale.com
Sent: Monday, November 09, 2009 4:24 PM
To: VSE Discussion List
Subject: Re: TCP/IP listeners

Post by Thomas Simonds
Is there any advantage to having multiple TCPIPSRVICE definitions
with the same IP address and port # when servicing a high volume of
concurrent transactions? I have a SOAP application that was
experiencing frequent connection resets and was unable to find the
reason. When I added a second listener the problem seemed to have
gone away.

My understanding is that you shouldn't (or, legitimately,
can't?) have more than one listener on the same IP address for the same
port. I'd be interested in any correction to my understanding. As for
high volumes, I've also been told that if the listener is written
correctly, you'd have to consistently have more than 64 simultaneous
connections attempted before you'd have trouble. Of course, in CICS,
you'd be in trouble, anyway, trying to handle that many spawned child
processes at the same time.

Sincerely,

Dave Clark

[David Stuart]

Hi Dave,

We're currently running the IBM TCP/IP Socket Listener. I have 8 defined for one Production application. During certain times of the year, all 8 can be in use.

It's not a problem, and I believe I set it up with IBM assistance and guidance.

Dave







Dave Stuart
Prin. Info. Systems Support Analyst
County of Ventura, CA
805-662-6731

Post by Thomas Simonds
Is there any advantage to having multiple TCPIPSRVICE definitions
with the same IP address and port # when servicing a high volume of
concurrent transactions? I have a SOAP application that was
experiencing frequent connection resets and was unable to find the
reason. When I added a second listener the problem seemed to have
gone away.

My understanding is that you shouldn't (or, legitimately, can't?)
have more than one listener on the same IP address for the same port. I'd
be interested in any correction to my understanding. As for high volumes,
I've also been told that if the listener is written correctly, you'd have
to consistently have more than 64 simultaneous connections attempted
before you'd have trouble. Of course, in CICS, you'd be in trouble,
anyway, trying to handle that many spawned child processes at the same
time.

Sincerely,

Dave Clark

WinWholesale Group Services
3110 Kettering Boulevard
Dayton, Ohio 45439 USA
[phone]


This email message and any attachments is for use only by the named
addressee(s) and may contain confidential, privileged and/or proprietary
information. If you have received this message in error, please
immediately notify the sender and delete and destroy the message and all
copies. All unauthorized direct or indirect use or disclosure of this
message is strictly prohibited. No right to confidentiality or privilege
is waived or lost by any error in transmission.

[i***@winwholesale.com]

Post by David Stuart
We're currently running the IBM TCP/IP Socket Listener. I have 8
defined for one Production application.

But I don't think the IBM CICS Listener is written "correctly" in
terms of the RFC standards in place. Meaning, there is actually only one
IBM listener listening at any given time. I've been told it is not
possible for a single TCP/IP stack to accept multiple listeners on the
same port. Basically, the IBM listener stops listening when it receives a
connection. If more than one listener is defined, then another listener
instance takes over while the first listener does whatever secondary
processing it has to do with the connection. Thus, even with multiple
possible IBM listeners defined for the same port, it is still possible for
the IBM listener to miss connections during the hand-off process. I've
been told that if a listener is written correctly, then it is always
listening as far as the TCP/IP stack is concerned and won't ever miss a
connection unless just too many simultaneous requests are present.

Sincerely,

Dave Clark

WinWholesale Group Services
3110 Kettering Boulevard
Dayton, Ohio 45439 USA
[phone]


This email message and any attachments is for use only by the named
addressee(s) and may contain confidential, privileged and/or proprietary
information. If you have received this message in error, please
immediately notify the sender and delete and destroy the message and all
copies. All unauthorized direct or indirect use or disclosure of this
message is strictly prohibited. No right to confidentiality or privilege
is waived or lost by any error in transmission.

[Tony Thigpen]

There are several things that are involved here.

The CSI SOCKET macro does not allow the type of "one
listener/multi-child" process that is normal in TCP/IP Listeners. You
need multiple listeners running because once a connection is made, that
socket is no longer available for more connections. Because of this, the
CSI stack must allow multiple listeners on the same port.

EZA, on the other hand, is designed for the "one listener/multi-child"
process that is normal on any other platform than VSE. In z/OS, it will
not normally allow multiple listeners on one port. (For now w are
ignoring a couple of special SETSOCKOPTs that they have to allow some
special processing.)

The CICS Listener is written in EZA, but when using the CSI stack, the
underlying code still uses SOCKET macros with it's restrictions. So
there is a lot of "multiple open SOCKETs' and 'generate a new SOCKET'
code in the EZA interface from IBM. If I remember correctly, they
generate 4 listeners. During heavy workloads, I could see where an extra
listener transaction would be very helpful.

Things are different for the BSI stack. We work more like z/OS. One
listener is all that is required for multiple connections. (Our EZA code
does not use the SOCKET macro.) However, we do deviate from the z/OS
"only one listener per port" rule. We had to make the special SETSOCKOPT
from z/OS the default because when we did not, IBM code had problems
because they expected a stack that would allow multiple listeners on one
port.

Clear as mud???

Tony Thigpen


-----Original Message -----
From: ***@winwholesale.com
Sent: 11/09/2009 05:34 PM

Post by i***@winwholesale.com

Post by David Stuart
We're currently running the IBM TCP/IP Socket Listener. I have 8
defined for one Production application.

But I don't think the IBM CICS Listener is written "correctly"
in terms of the RFC standards in place. Meaning, there is actually only
one IBM listener listening at any given time. I've been told it is not
possible for a single TCP/IP stack to accept multiple listeners on the
same port. Basically, the IBM listener stops listening when it receives
a connection. If more than one listener is defined, then another
listener instance takes over while the first listener does whatever
secondary processing it has to do with the connection. Thus, even with
multiple possible IBM listeners defined for the same port, it is still
possible for the IBM listener to miss connections during the hand-off
process. I've been told that if a listener is written correctly, then
it is always listening as far as the TCP/IP stack is concerned and won't
ever miss a connection unless just too many simultaneous requests are
present.
Sincerely,
Dave Clark
WinWholesale Group Services
3110 Kettering Boulevard
Dayton, Ohio 45439 USA
[phone]
This email message and any attachments is for use only by the named
addressee(s) and may contain confidential, privileged and/or proprietary
information. If you have received this message in error, please
immediately notify the sender and delete and destroy the message and all
copies. All unauthorized direct or indirect use or disclosure of this
message is strictly prohibited. No right to confidentiality or privilege
is waived or lost by any error in transmission.

[Ed Franks]

In terms of TCP/IP in general and the IBM/CSI VSE stack in particular:

Each TCP/IP connection starts with one "listener" which sits on a
particular IP address and port. When a remote client requests the
matching IP address and port, the two are matched-up and a connection is
established. If a second client simultaneously requests the same IP and
port, its request is rejected because the "listener" was "used up" by
the first client. In most cases, the server side will quickly establish
another "listener", either when the current transaction is finished or
as soon a the "listener" is converted to an established connection.

The best practice is to maintain multiple "listeners" so that one is
always available to field every client's request. There is no limit
(other than practical) on the number of "listeners" for any given IP
address and port combination. The only consideration is that you have
no control over which listener will be used first, so all listeners on a
port must do the same thing.

The method for creating multiple listeners depends on the stack API
being used.

If your server application is using the BSD API, then the BSD overhead
layer will perform listener overlap to whatever degree the application
requests.

If your server application is written at the more basic "socket" level,
then you have two choices:

1. Issue multiple passive OPENs to match the level of concurrent
connections you desire.

2. Tell the stack (via the PORTQUEUE command) how many connection
requests it should queue.

When port queuing is in effect (for a given port) and there is no
"listener" available, the stack will create a "pseudo listener" each
time a connection request is received. When the local server
application eventually does issue a "listen" it is immediately matched
with a oldest queued connection request. Note that this activity is
invisible to both the client and server, so it will work with any API.

You should also keep in mind that not all client applications behave in
what mainframers would consider to be a reasonable manner. For example,
it is not uncommon to see a web browser issue three simultaneous open
requests, use the first one established, and then abandon the other two.
It may also hold one or more of the connections open for 10 or fifteen
minutes before allowing it to close.

If anyone wants more detailed information, please contact me off-list
with particular questions.

Ed Franks

[Don Stoever]

The CSI stack allows a single listener or multiple listeners on the port.
It is imho a application design consideration.

Our FTP server uses the socket macro with single or multiple listeners.
The ftp server can use either internal pseudo tasks in the TCP/IP partition(64k max pseudo
tasks),
or real VSE subtasks using the external ftpbatch in server mode.
When a new FTP client connects into the re-entrant FTP server,
it immediately dispatches a pre-attached subtask to handle the new connection.
The server then re-enters the listen state.
Now this very small time period before the server re-enters the listen state, can result in the
syn from a new client connection being rejected.
This is the same as if you attempted to connect to a unused port.
The DIAG CONNREJ can be issued to see when or if any client connections are being rejected.
You could then either issue a 2nd DEFINE FTPD with the same port and have multiple ftp servers
listening on the same port, or have a external ftp server with // exec ftpbatch running, again
all listening on the same port.
The ftpbatch is interesting because you could either:

// EXEC FTPBATCH,SIZE=FTPBATCH,PARM='FTPDPORT=21,MAXACT=8'
and have a max of 8 ftp sessions serviced in a single 32meg or larger partition.
Q CONN,LPORT=21 would show 1 listen connection on port 21.

// EXEC FTPBATCH,SIZE=FTPBATCH,PARM='FTPDPORT=21,MAXACT=2'
and have a max of 2 ftp sessions serviced in 4 8meg or larger partitions.
Q CONN,LPORT=21 would show 4 listen connections on port 21.
This means you would have the ftpbatch running in 4 partitions at the same time.

Our BSD interface also fully supports the backlog parameter on a listen request.
This is the 2nd parm passed in from a application and specifies the maximum number of incoming
connection requests to be queued. These would be syn requests with no listening server for the
window of time the application is processing a new connection before it re-enters the listen
state.
We also allow this to be controlled with our $sockopt options phase.
This queing on incoming connection requests(tcp syn's) can also be controlled with the PORTQ
command to allow any server application written in any api to queue inbound connection
requests(syn's).
But if you are wondering if any of this is necessary first issue a DIAG CONNREJ to see if you
are having any inbound connection requests rejected.

[David Stuart]

Tom,

We set up the TCP/IP Listeners for a vendor product several years ago. For several days, we had a problem with connection time-outs and connections pending. Eventually, with some guidance, I added more listeners, and all the problems went away. This was using just one TCPIPSERVICE definition.

I think the only time you would need to define another TCPIPSERVICE, is if you wanted/needed to define listeners that were listening on a different port from the already existing one, as the port is specified in the TCPIPSERVICE definitions.


Dave







Dave Stuart
Prin. Info. Systems Support Analyst
County of Ventura, CA
805-662-6731
***@ventura.org>>> "Thomas Simonds" <***@cogsdale.com> 11/9/2009 12:58 PM >>>
Is there any advantage to having multiple TCPIPSRVICE definitions with the same IP address and port # when servicing a high volume of concurrent transactions? I have a SOAP application that was experiencing frequent connection resets and was unable to find the reason. When I added a second listener the problem seemed to have gone away. I wondered if anyone else had experience with this.

Tom Simonds
Cogsdale Corporation
703-727-2678