Recently meet an SCTP issue: one product application running on MontaVista(kernel version 2.6.21) can't run on RedHat Enterprise Linux 5.5. After hours investion between millions lines  of  log file and source code, I finally noticed following error showed up:

	^?0 2011-08-16 09:11:57.316267 coolnjmcl018 AMMServ 26555[26566]

	 ../com/AMMSSctpLib.cc,369 [EAMMS1315] AMMSSctpLib::ammsSetSockOpt()

	 setsockopt - Invalid argument

	^?1 2011-08-16 09:11:57.316326 coolnjmcl018 AMMServ 26555[26566]

	../com/AMMSAccessServer.cc,779 [EAMMS1036] AMMSAccessServer::setSocketOpts():

	 failed to set SCTP_EVENTS for socket 6
C function ammsSetSockOpt() was a function used in that product application, which eventually calls Linux system API setsockopt().

The product developers told me that they updated libsctp.so shared library in MV, so I copied that library directly to RHEL5.5(usually this way will work). However, that shared library didn't work this time.

Then I noticed that the kernel of RHEL5.5 was 2.6.18-194.el5, which means Linux kernel 2.6.18 was used. I also found some webpages about RHEL kernel, and all of them claimed that Red Hat would backport new-developed Linux kernel(community) features to RHEL5 kernels, which means that the older kernel version of RHEL may also contains some features in higher Linux kernel.  Since SCTP codes were largely changed in 2.6.21, I decided to upgrade RHEL kernel to 2.6.18-238.9.1.el5(the default version in RHEL5.7).

After 4.5 hours compiling and installing, RHEL kernel upgraded to the latest one. Unfortunately, the SCTP issue still existed.

I wen to back to reading source code, and tried to change the last parameter of setsockopt() from sizeof(events) to 8, and it worked!  Where, events is defined as

	struct sctp_event_subscribe events;
Then I began to check the definition of struct sctp_event_subscribe in both RHEL5.5 and RHEL6.0 - they were different:

In RHEL 6,

 402/*
 403 * Described in Section 7.3
 404 *   Ancillary Data and Notification Interest Options
 405 */
 406struct sctp_event_subscribe {
 407        __u8 sctp_data_io_event;
 408        __u8 sctp_association_event;
 409        __u8 sctp_address_event;
 410        __u8 sctp_send_failure_event;
 411        __u8 sctp_peer_error_event;
 412        __u8 sctp_shutdown_event;
 413        __u8 sctp_partial_delivery_event;
 414        __u8 sctp_adaptation_layer_event;
 415        __u8 sctp_authentication_event;
 416};

While in RHEL5,

 363/*
 364 * Described in Section 7.3
 365 *   Ancillary Data and Notification Interest Options
 366 */
 367struct sctp_event_subscribe {
 368        __u8 sctp_data_io_event;
 369        __u8 sctp_association_event;
 370        __u8 sctp_address_event;
 371        __u8 sctp_send_failure_event;
 372        __u8 sctp_peer_error_event;
 373        __u8 sctp_shutdown_event;
 374        __u8 sctp_partial_delivery_event;
 375        __u8 sctp_adaption_layer_event;
 376};

It seems upgrading RHEL5.5 to RHEL6 is the final choice!