Protect yourself against future threats.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
===========================================================================
AUSCERT External Security Bulletin Redistribution
ESB-2009.1479
Solaris 10 Kernel Patches 141444-09 and 141445-09 May Cause
Interface Failure in IP Multipathing (IPMP)
5 November 2009
===========================================================================
AusCERT Security Bulletin Summary
---------------------------------
Product: Kernel
Publisher: Sun Microsystems
Operating System: Solaris
Impact/Access: Denial of Service -- Existing Account
Resolution: Mitigation
Original Bulletin:
http://sunsolve.sun.com/search/printfriendly.do?assetkey=1-66-271519-1
- --------------------------BEGIN INCLUDED TEXT--------------------
Solaris 10 Kernel Patches 141444-09 and 141445-09 May Cause Interface
Failure in IP Multipathing (IPMP)
_________________________________________________________________
Category : Availability
Release Phase : Workaround
Bug Id : 6888928
Product : Solaris 10 Operating System
Date of Workaround Release : 03-Nov-2009
Solaris 10 Kernel Patches 141444-09 and 141445-09 cause interface failure
in IP Multipathing:
1. Impact
The IP Multipathing (IPMP) facility allows systems with multiple
network interfaces in the same subnet to failover in the event that
one of the interfaces fails, and then failback when the interface is
ok.
Solaris 10 Kernel Patches 141444-09 (SPARC) and 141445-09 (x86)
cause interface failure in IPMP when configured for probe based
failure detection. This issue does not occur with a IPMP link based
failure detection configuration.
2. Contributing Factors
This issue can occur in the following releases:
SPARC Platform:
* Solaris 10 with patch 141444-09
x86 Platform:
* Solaris 10 with patch 141445-09
Note 1: Solaris 8 and 9 and OpenSolaris are not impacted by this
issue.
Note 2: This issue only occurs on systems with IPMP configured for
probe based failure detection. This issue does not occur with a IPMP
link based failure detection configuration.
To determine if IPMP is configured for probe based failure detection,
all the following must be true:
1. The "in.mpathd" daemon is running as shown by the following
command:
# ps -aef |grep "in.mpathd"
root 211 1 0 11:04:51 ? 0:00 /usr/lib/inet/in.mpathd -a
2. The output from "ifconfig -a" must show "groupname" for the
interfaces in the IPMP group as shown below:
# ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8
232 index 1
inet 127.0.0.1 netmask ff000000
e1000g1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500
index 5
inet 192.178.100.1 netmask ffffff00 broadcast 192.178.100.255
groupname fred
ether 0:3:ba:d8:d1:ef
e1000g1:1: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,
NOFAILOVER> mtu 1500 index 5
inet 192.178.100.2 netmask ffffff00 broadcast 192.178.100.255
e1000g2: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500
index 6
inet 192.178.100.5 netmask ffffff00 broadcast 192.178.100.255
groupname fred
ether 0:4:23:c8:33:86
e1000g2:1: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,
NOFAILOVER> mtu 1500 index 6
inet 192.178.100.6 netmask ffffff00 broadcast 192.178.100.255
In the example above, e1000g1 and e1000g2 are part of IPMP group
called "fred".
3. The output of "ifconfig -a" must show "DEPRECATED" and
"NOFAILOVER" for test addresses, as this indicates that IPMP probe
based failure detection configuration is being used. This is shown in
the output above for both e1000g1 and e1000g2.
In the above example, 192.178.100.2 (address on e1000g1:1) and
192.178.100.6 (address on e1000g2:1) are test addresses.
3. Symptoms
If the described issue occurs, an interface in the IPMP group fails
even though no network problems have been experienced.
The following messages are printed on the console (or
/var/adm/messages):
# Oct 22 11:09:29 v4v-t2000a-sca11 in.mpathd: NIC failure detected on
e1000g2 of group fred
Oct 22 11:09:29 v4v-t2000a-sca11 in.mpathd: Successfully failed over
from NIC e1000g2 to NIC e1000g1
And the interface is marked FAILED in the output of "ifconfig -a" as
shown below:
# ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232
index 1
inet 127.0.0.1 netmask ff000000
e1000g1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500
index 5
inet 192.178.100.1 netmask ffffff00 broadcast 192.178.100.255
groupname fred
ether 0:3:ba:d8:d1:ef
e1000g1:1: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,
NOFAILOVER> mtu 1500 index 5
inet 192.178.100.2 netmask ffffff00 broadcast 192.178.100.255
e1000g1:2: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500
index 5
inet 192.178.100.5 netmask ffffff00 broadcast 192.178.100.255
e1000g2: flags=19000842<BROADCAST,RUNNING,MULTICAST,IPv4,NOFAILOVER,FAILED
> mtu 0 index 6
inet 0.0.0.0 netmask 0
groupname fred
ether 0:4:23:c8:33:86
e1000g2:1: flags=19040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,
NOFAILOVER,FAILED> mtu 1500 index 6
inet 192.178.100.6 netmask ffffff00 broadcast 192.178.100.255
In the example above, e1000g2 is marked as FAILED.
The interface failed because the ICMP probe replies for 192.178.100.6
are received on e1000g1 instead of e1000g2.
Snooping on e1000g1 reveals that 192.178.100.6 ICMP echo requests are
sent from e1000g1 and also ICMP echo replies for 192.178.100.6 are
received on e1000g1:
# snoop -d e1000g1 icmp
Using device e1000g1 (promiscuous mode)
192.178.100.6 -> 192.178.100.15 ICMP Echo request (ID: 54022 Sequence
number: 1674)
192.178.100.15 -> 192.178.100.6 ICMP Echo reply (ID: 54022 Sequence number:
1674)
192.178.100.2 -> 192.178.100.15 ICMP Echo request (ID: 54021 Sequence
number: 1680)
192.178.100.15 -> 192.178.100.2 ICMP Echo reply (ID: 54021 Sequence number:
1680)
192.178.100.6 -> 192.178.100.10 ICMP Echo request (ID: 54022 Sequence
number: 1675)
192.178.100.10 -> 192.178.100.6 ICMP Echo reply (ID: 54022 Sequence number:
1675)
Snooping on e1000g2 (shown below) indicates that the FAILED e1000g2
interface is not sending or receiving any probe packets. However, the
e1000g2 interface should send probe packets even if the interface has
failed. This is another symptom of this issue.
# snoop -d e1000g2 icmp
Using device e1000g2 (promiscuous mode)
4. Workaround
Binary relief is available through normal support channels.
Note: Removing the offending patches to avoid this issue is not
advisable as these patches contain security fixes.
5. Resolution
A final resolution is pending completion.
This Sun Alert notification is being provided to you on an "AS IS"
basis. This Sun Alert notification may contain information provided by
third parties. The issues described in this Sun Alert notification may
or may not impact your system(s). Sun makes no representations,
warranties, or guarantees as to the information contained herein. ANY
AND ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING WITHOUT LIMITATION
WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR
NON-INFRINGEMENT, ARE HEREBY DISCLAIMED. BY ACCESSING THIS DOCUMENT
YOU ACKNOWLEDGE THAT SUN SHALL IN NO EVENT BE LIABLE FOR ANY DIRECT,
INDIRECT, INCIDENTAL, PUNITIVE, OR CONSEQUENTIAL DAMAGES THAT ARISE
OUT OF YOUR USE OR FAILURE TO USE THE INFORMATION CONTAINED HEREIN.
This Sun Alert notification contains Sun proprietary and confidential
information. It is being provided to you pursuant to the provisions of
your agreement to purchase services from Sun, or, if you do not have
such an agreement, the Sun.com Terms of Use. This Sun Alert
notification may only be used for the purposes contemplated by these
agreements.
Copyright 2000-2009 Sun Microsystems, Inc., 4150 Network Circle, Santa
Clara, CA 95054 U.S.A. All rights reserved
- --------------------------END INCLUDED TEXT--------------------
You have received this e-mail bulletin as a result of your organisation's
registration with AusCERT. The mailing list you are subscribed to is
maintained within your organisation, so if you do not wish to continue
receiving these bulletins you should contact your local IT manager. If
you do not know who that is, please send an email to auscert@auscert.org.au
and we will forward your request to the appropriate person.
NOTE: Third Party Rights
This security bulletin is provided as a service to AusCERT's members. As
AusCERT did not write the document quoted above, AusCERT has had no control
over its content. The decision to follow or act on information or advice
contained in this security bulletin is the responsibility of each user or
organisation, and should be considered in accordance with your organisation's
site policies and procedures. AusCERT takes no responsibility for consequences
which may arise from following or acting on information or advice contained in
this security bulletin.
NOTE: This is only the original release of the security bulletin. It may
not be updated when updates to the original are made. If downloading at
a later date, it is recommended that the bulletin is retrieved directly
from the author's website to ensure that the information is still current.
Contact information for the authors of the original document is included
in the Security Bulletin above. If you have any questions or need further
information, please contact them directly.
Previous advisories and external security bulletins can be retrieved from:
http://www.auscert.org.au/render.html?cid=1980
If you believe that your computer system has been compromised or attacked in
any way, we encourage you to let us know by completing the secure National IT
Incident Reporting Form at:
http://www.auscert.org.au/render.html?it=3192
===========================================================================
Australian Computer Emergency Response Team
The University of Queensland
Brisbane
Qld 4072
Internet Email: auscert@auscert.org.au
Facsimile: (07) 3365 7031
Telephone: (07) 3365 4417 (International: +61 7 3365 4417)
AusCERT personnel answer during Queensland business hours
which are GMT+10:00 (AEST).
On call after hours for member emergencies only.
===========================================================================
-----BEGIN PGP SIGNATURE-----
Comment: http://www.auscert.org.au/render.html?it=1967
iD8DBQFK8hmZNVH5XJJInbgRAhBaAJ94B2nwDhMGPKwryeRqpdxCOfYguACdGbGi
k4jsHjveWtrh1VvZMUxaZ0E=
=GViB
-----END PGP SIGNATURE-----