On Tue, Oct 11, 2011 at 12:17 PM, Matthew Brett wrote:
> Hi,
>
> While struggling with floating point precision, I ran into this:
>
> In [52]: a = 2**54+3
>
> In [53]: a
> Out[53]: 18014398509481987L
>
> In [54]: np.float128(a)
> Out[54]: 18014398509481988.0
>
> In [55]: np.float128(a)-1
> Out[55]
On 10/12/11, "V. Armando Solé" wrote:
> On 12/10/2011 10:46, David Cournapeau wrote:
>> On Wed, Oct 12, 2011 at 9:18 AM, "V. Armando Solé" wrote:
>>> From a pure user perspective, I would not expect the abs function to
>>> return a negative number. Returning +127 plus a warning the first time
>>
On 11/10/11 21:16, Charles R Harris wrote:
> IIRC, matlab was said to return
> +127 as abs(-128), which, if true, is quite curious.
I just checked and this is indeed the case in Matlab 7.10.0 R2010a:
>> abs(int8(-128))
ans =
127
Cheers,
--
Daniele
___
On 12/10/2011 10:46, David Cournapeau wrote:
> On Wed, Oct 12, 2011 at 9:18 AM, "V. Armando Solé" wrote:
>> From a pure user perspective, I would not expect the abs function to
>> return a negative number. Returning +127 plus a warning the first time
>> that happens seems to me a good compromise.
On Wed, Oct 12, 2011 at 9:18 AM, "V. Armando Solé" wrote:
> From a pure user perspective, I would not expect the abs function to
> return a negative number. Returning +127 plus a warning the first time
> that happens seems to me a good compromise.
I guess the question is what's the common contex
From a pure user perspective, I would not expect the abs function to
return a negative number. Returning +127 plus a warning the first time
that happens seems to me a good compromise.
Armando
On 12/10/2011 09:46, David Cournapeau wrote:
> On Tue, Oct 11, 2011 at 8:16 PM, Charles R Harris
> w
On Tue, Oct 11, 2011 at 8:16 PM, Charles R Harris
wrote:
>
>
> On Tue, Oct 11, 2011 at 12:23 PM, Matthew Brett
> wrote:
>>
>> Hi,
>>
>> I recently ran into this:
>>
>> In [68]: arr = np.array(-128, np.int8)
>>
>> In [69]: arr
>> Out[69]: array(-128, dtype=int8)
>>
>> In [70]: np.abs(arr)
>> Out[7
