Bug 37028

GemBuilder for Smalltalk/VW

7.1.2, 7.1.1, 7.1, 7.0.2, 7.0.1, 7.0

GS64

7.2

SmallDouble and Double problems on fault/flush

When running with GemStone/S 64 Bit v.2.2x, a number of problems
occur when faulting or flushing Doubles and server DmallDoubles at
the farther ranges of values.

(1) GBS raised an error on faulting SmallDoubles whose exponents
were 0xFF. The range of numbers affected for this case was from:
  (2.0 raisedTo: 128)
to:
  (2.0 raisedTo: 129) - (2.0 raisedTo: 76)
inclusive (postive and negative numbers).


(2) Incorrect values were created on faulting non-zero SmallDoubles
whose exponents were 0x00. These came in as subnormal Doubles,
with values much closer to zero than they should have been.

The range of numbers affected by this case was from:
  (2.0 raisedTo: -127) + (2.0 raisedTo: -179)
to
  (2.0 raisedTo: -126) - (2.0 raisedTo: -179)
postive and negative numbers.

(3) Flushing of client INFs and NaNs (which should be rare since
VW does not fully support them yet) were incorrectly flushed as
server SmallDoubles in the range:
  (2.0 raisedTo: 128)
to:
  (2.0 raisedTo: 129) - (2.0 raisedTo: 76)

(4) Some non-zero client subnormal Doubles were flushed in the
incorrect range of SmallDoubles, becaming much farther than zero
than is correct. The range of client values affected, both positive
and negative:
  (2.0d0 raisedTo: -1074)
to
  (2.0d0 raisedTo: -1022) - (2.0d0 raisedTo: -1074)

Workaround

GbxSpecialConverter64>>localFromSmallDoubleOop:


localFromSmallDoubleOop: delegate
     "delegate is known to represent a SmallDouble. Answer the equivalent local Double."

     | double exp8 exp11 sign |
     exp8 := delegate oopByteAt: 1.     "Fetch the 8-bit exponent of the SmallDouble"
     exp11 := exp8 + 16r380.     "Compute the Double's 11-bit exponent."
     sign := (delegate oopByteAt: 8) bitAnd: 8.
     double := Double basicNew: 8.     "VW7 always makes it 8 bytes, but you have to tell VW5i."
     double basicAt: 8 put: ((sign bitShift: 4) bitOr: (exp11 bitShift: -4)).
     double basicAt: 7
          put: (((exp11 bitAnd: 16r0F) bitShift: 4)
                    bitOr: ((delegate oopByteAt: 2) bitShift: -4)).
     double basicAt: 6
          put: ((((delegate oopByteAt: 2) bitAnd: 16r0F) bitShift: 4)
                    bitOr: ((delegate oopByteAt: 3) bitShift: -4)).
     double basicAt: 5
          put: ((((delegate oopByteAt: 3) bitAnd: 16r0F) bitShift: 4)
                    bitOr: ((delegate oopByteAt: 4) bitShift: -4)).
     double basicAt: 4
          put: ((((delegate oopByteAt: 4) bitAnd: 16r0F) bitShift: 4)
                    bitOr: ((delegate oopByteAt: 5) bitShift: -4)).
     double basicAt: 3
          put: ((((delegate oopByteAt: 5) bitAnd: 16r0F) bitShift: 4)
                    bitOr: ((delegate oopByteAt: 6) bitShift: -4)).
     double basicAt: 2
          put: ((((delegate oopByteAt: 6) bitAnd: 16r0F) bitShift: 4)
                    bitOr: ((delegate oopByteAt: 7) bitShift: -4)).
     double basicAt: 1
          put: ((((delegate oopByteAt: 7) bitAnd: 16r0F) bitShift: 4)
                    bitOr: ((delegate oopByteAt: 8) bitShift: -4)).


     "The above assumes little-endian platform. If we're on a big-endian platform, correct it."
     self isBigEndian ifTrue: [double reverseByteOrder].

     "Check for zero. If zero, we will have converted it to +-2^-127, represented here as a
     constant to avoid computing it each time."
     double abs = 5.877471754111438d-39 ifTrue: [double := 0.0d0].
     "Check for Infs and NaNs"
     GBSM checkFloat: double.
     ^double


------------------------------

GbxSpecialConverter64>>delegateForSpecialDouble:forSession:

delegateForSpecialDouble: aDouble forSession: aSession
     "Answer nil if aDouble is not in the range for server SmallDoubles.
     Otherwise, answer an existing or new delegate representing the server SmallDouble."

     | del double highByte sign exp11 exp8 |

     "There is one positive and its corresponding negative Double that would normally be able to be a SmallDouble,
     but that bit pattern is used for zero. Catch those up front."
     aDouble abs = 5.877471754111438d-39 ifTrue: [^nil].
     "Get the source double in little-endian byte ordering."
     double := self isBigEndian
                    ifTrue: [aDouble copy reverseByteOrder]
                    ifFalse: [aDouble].
     highByte := double basicAt: 8.
     sign := highByte bitAnd: 16r80.     "sign is 0 (positive) or 128 (negative)"
     exp11 := ((highByte bitAnd: 16r7F) bitShift: 4)
                    bitOr: ((double basicAt: 7) bitShift: -4).     "Fetch the 11-bit exponent of the Double."
     "0.0 is a special case. Although VW doesn't seem to yet support negative 0.0, I test here for it anyway."
     exp8 := aDouble abs = 0.0d0 ifTrue: [16r00] ifFalse: [exp11 - 16r380].     "Compute the SmallDouble's 8-bit exponent."
     (exp8 between: 0 and: 16rFF) ifFalse: [^nil].     "Out of range."
     del := GbxDelegate forSession: aSession.
     del oopByteAt: 1 put: exp8.
     del oopByteAt: 2
          put: ((((double basicAt: 7) bitAnd: 16r0F) bitShift: 4)
                    bitOr: ((double basicAt: 6) bitShift: -4)).
     del oopByteAt: 3
          put: ((((double basicAt: 6) bitAnd: 16r0F) bitShift: 4)
                    bitOr: ((double basicAt: 5) bitShift: -4)).
     del oopByteAt: 4
          put: ((((double basicAt: 5) bitAnd: 16r0F) bitShift: 4)
                    bitOr: ((double basicAt: 4) bitShift: -4)).
     del oopByteAt: 5
          put: ((((double basicAt: 4) bitAnd: 16r0F) bitShift: 4)
                    bitOr: ((double basicAt: 3) bitShift: -4)).
     del oopByteAt: 6
          put: ((((double basicAt: 3) bitAnd: 16r0F) bitShift: 4)
                    bitOr: ((double basicAt: 2) bitShift: -4)).
     del oopByteAt: 7
          put: ((((double basicAt: 2) bitAnd: 16r0F) bitShift: 4)
                    bitOr: ((double basicAt: 1) bitShift: -4)).
     del oopByteAt: 8
          put: ((((double basicAt: 1) bitAnd: 16r0F) bitShift: 4)
                    bitOr: ((sign bitShift: -4) bitOr: 2r110)).     "110 is theSmallDouble tag"
     ^del


Last updated: 10/5/07