On Mon, Jun 13, 2011 at 11:59 AM, wrote:
> I'm joining this late (I've been traveling), but it might be useful to
> look at the fairly new R module "lubridate". They have put quite some
> thought into simplifying date handling, and when I have used it I have
> generally been quite pleased. The do
On Fri, Jun 10, 2011 at 7:04 PM, Pierre GM wrote:
>
> On Jun 11, 2011, at 1:03 AM, Mark Wiebe wrote:
> >
> > I don't think you would want to extend the datetime with more metadata,
> but rather use it as a tool to create the timeseries with. You could create
> a lightweight wrapper around datetim
I'm joining this late (I've been traveling), but it might be useful to
look at the fairly new R module "lubridate". They have put quite some
thought into simplifying date handling, and when I have used it I have
generally been quite pleased. The documentation is quite readable.
Just Google it and
On Jun 11, 2011, at 1:03 AM, Mark Wiebe wrote:
>
> I don't think you would want to extend the datetime with more metadata, but
> rather use it as a tool to create the timeseries with. You could create a
> lightweight wrapper around datetime arrays which exposed a
> timeseries-oriented interfac
On Thu, Jun 9, 2011 at 3:17 PM, Wes McKinney wrote:
> On Wed, Jun 8, 2011 at 8:53 PM, Mark Wiebe wrote:
> > On Wed, Jun 8, 2011 at 4:57 AM, Wes McKinney
> wrote:
> >>
> >>
> >>
> >>
> >> So in summary, w.r.t. time series data and datetime, the only things I
> >> care about from a datetime / pa
On Thu, Jun 9, 2011 at 1:27 PM, Christopher Barker wrote:
> Mark Wiebe wrote:
> > Because datetime64 is a NumPy data type, it needs a well-defined rule
> > for these kinds of conversions. Treating datetimes as moments in time
> > instead of time intervals makes a very nice rule which appears to be
On Thu, Jun 9, 2011 at 1:44 AM, Pierre GM wrote:
> >
> > The fact that it's a NumPy dtype probably is the biggest limiting factor
> preventing parameters like 'start' and 'end' during conversion. Having a
> datetime represent an instant in time neatly removes any ambiguity, so
> converting betwee
On Wed, Jun 8, 2011 at 8:53 PM, Mark Wiebe wrote:
> On Wed, Jun 8, 2011 at 4:57 AM, Wes McKinney wrote:
>>
>>
>>
>>
>> So in summary, w.r.t. time series data and datetime, the only things I
>> care about from a datetime / pandas point of view:
>>
>> - Ability to easily define custom timedeltas
>
Mark Wiebe wrote:
> Because datetime64 is a NumPy data type, it needs a well-defined rule
> for these kinds of conversions. Treating datetimes as moments in time
> instead of time intervals makes a very nice rule which appears to be
> very amenable to a variety of computations, which is why I li
>
> The fact that it's a NumPy dtype probably is the biggest limiting factor
> preventing parameters like 'start' and 'end' during conversion. Having a
> datetime represent an instant in time neatly removes any ambiguity, so
> converting between days and seconds as a unit is analogous to conver
On Wed, Jun 8, 2011 at 4:57 AM, Wes McKinney wrote:
>
>
>
> So in summary, w.r.t. time series data and datetime, the only things I
> care about from a datetime / pandas point of view:
>
> - Ability to easily define custom timedeltas
>
Can you elaborate on this a bit? I'm guessing you're not ref
On Wed, Jun 8, 2011 at 5:59 AM, Dave Hirschfeld
wrote:
> Mark Wiebe gmail.com> writes:
> >
> >
> > It appears to me that a structured dtype with some further NumPy
> extensions
> > could entirely replace the 'events' metadata fairly cleanly. If the
> ufuncs
> > are extended to operate on structur
On Tue, Jun 7, 2011 at 7:28 PM, Pierre GM wrote:
> >
> > It supports .astype(), with a truncation policy. This is motivated
> partially because that's how Pythons integer division works, and partially
> because if you consider a full datetime '2011-03-14T13:22:16', it's natural
> to think of the
Mark Wiebe gmail.com> writes:
>
>
> It appears to me that a structured dtype with some further NumPy extensions
> could entirely replace the 'events' metadata fairly cleanly. If the ufuncs
> are extended to operate on structured arrays, and integers modulo n are
> added as a new dtype, a dtyp
On Wed, Jun 8, 2011 at 6:37 AM, Dave Hirschfeld
wrote:
> Wes McKinney gmail.com> writes:
>
>>
>> >
>> > - Fundamental need to be able to work with multiple time series,
>> > especially performing operations involving cross-sectional data
>> > - I think it's a bit hard for lay people to use (read:
Wes McKinney gmail.com> writes:
>
> >
> > - Fundamental need to be able to work with multiple time series,
> > especially performing operations involving cross-sectional data
> > - I think it's a bit hard for lay people to use (read: ex-MATLAB/R
> > users). This is just my opinion, but a few yea
On Wed, Jun 8, 2011 at 11:57 AM, Wes McKinney wrote:
> On Wed, Jun 8, 2011 at 7:36 AM, Chris Barker wrote:
>> On 6/7/11 4:53 PM, Pierre GM wrote:
>>> Anyhow, each time yo
>>> read 'frequency' in scikits.timeseries, think 'unit'.
>>
>> or maybe "precision" -- when I think if unit, I think of some
On Wed, Jun 8, 2011 at 7:36 AM, Chris Barker wrote:
> On 6/7/11 4:53 PM, Pierre GM wrote:
>> Anyhow, each time yo
>> read 'frequency' in scikits.timeseries, think 'unit'.
>
> or maybe "precision" -- when I think if unit, I think of something that
> can be represented as a floating point value --
On 6/7/11 4:53 PM, Pierre GM wrote:
> Anyhow, each time yo
> read 'frequency' in scikits.timeseries, think 'unit'.
or maybe "precision" -- when I think if unit, I think of something that
can be represented as a floating point value -- but here, with integers,
it's the precision that can be repr
>
> It supports .astype(), with a truncation policy. This is motivated partially
> because that's how Pythons integer division works, and partially because if
> you consider a full datetime '2011-03-14T13:22:16', it's natural to think of
> the year as '2011', the date as '2011-03-14', etc, whic
On Tue, Jun 7, 2011 at 6:53 PM, Pierre GM wrote:
>
> On Jun 8, 2011, at 1:16 AM, Mark Wiebe wrote:
>
> > Hi Dave,
> >
> > Thanks for all the feedback on the datetime, it's very useful to help
> understand the timeseries ideas, in particular with the many examples you're
> sprinkling in.
> >
> > O
On Jun 8, 2011, at 1:16 AM, Mark Wiebe wrote:
> Hi Dave,
>
> Thanks for all the feedback on the datetime, it's very useful to help
> understand the timeseries ideas, in particular with the many examples you're
> sprinkling in.
>
> One overall impression I have about timeseries in general is t
On Tue, Jun 7, 2011 at 3:56 PM, Pierre GM wrote:
>
> On Jun 7, 2011, at 5:54 PM, Robert Kern wrote:
>
> > On Tue, Jun 7, 2011 at 07:34, Dave Hirschfeld
> wrote:
> >
> >> I'm not convinced about the events concept - it seems to add complexity
> >> for something which could be accomplished better
Hi Dave,
Thanks for all the feedback on the datetime, it's very useful to help
understand the timeseries ideas, in particular with the many examples you're
sprinkling in.
One overall impression I have about timeseries in general is the use of the
term "frequency" synonymously with the time unit.
Christopher Barker noaa.gov> writes:
>
> Dave Hirschfeld wrote:
> > That would be one way of dealing with irregularly spaced data. I would argue
> > that the example is somewhat back-to-front though. If something happens
> > twice a month it's not occuring at a monthly frequency, but at a higher
On Jun 7, 2011, at 5:54 PM, Robert Kern wrote:
> On Tue, Jun 7, 2011 at 07:34, Dave Hirschfeld
> wrote:
>
>> I'm not convinced about the events concept - it seems to add complexity
>> for something which could be accomplished better in other ways. A [Y]//4
>> dtype is better specified as [3M]
Dave Hirschfeld wrote:
> Robert Kern gmail.com> writes:
>> Well, [D/100] doesn't represent [864s]. It represents something that
>> happens 100 times a day, but not necessarily at precise regular
>> intervals.
>> For example, suppose that I am representing payments that
>> happen twice a month, s
Robert Kern gmail.com> writes:
>
> On Tue, Jun 7, 2011 at 07:34, Dave Hirschfeld gmail.com>
wrote:
>
> > I'm not convinced about the events concept - it seems to add complexity
> > for something which could be accomplished better in other ways. A [Y]//4
> > dtype is better specified as [3M] dt
Pierre GM wrote:
> Using the ISO as reference, you have a good definition of months.
Yes, but only one. there are others. For instance, the climate modelers
like to use a calendar that has 360 days a year: 12 30 day months. That
way they get something with the same timescale as months and years,
On Tue, Jun 7, 2011 at 9:54 AM, Robert Kern wrote:
> On Tue, Jun 7, 2011 at 07:34, Dave Hirschfeld
> wrote:
>
> > I'm not convinced about the events concept - it seems to add complexity
> > for something which could be accomplished better in other ways. A [Y]//4
> > dtype is better specified as
On Tue, Jun 7, 2011 at 07:34, Dave Hirschfeld wrote:
> I'm not convinced about the events concept - it seems to add complexity
> for something which could be accomplished better in other ways. A [Y]//4
> dtype is better specified as [3M] dtype, a [D]//100 is an [864S]. There
> may well be a good
As a user of numpy/scipy in finance I thought I would put in my 2p worth as
it's something which is of great importance in this area.
I'm currently a heavy user of the scikits.timeseries package by Matt & Pierre
and I'm also following the development of statsmodels and pandas should we
require m
On Mon, Jun 6, 2011 at 2:31 PM, Mark Wiebe wrote:
> On Mon, Jun 6, 2011 at 1:33 PM, Christopher Barker
> wrote:
>
>>
>>
> This is even self inconsistent:
>>
>> 1Y == 365D
>>
>> 1Y == 12M == 12 * 30D == 360D
>>
>> 1Y == 12M == 12 * 4W == 12 * 4 * 7D == 336D
>>
>> 1Y == 52W == 52 * 7D == 364D
>>
On Jun 6, 2011, at 8:33 PM, Christopher Barker wrote:
> Mark Wiebe wrote:
>> I'm wondering if removing the business-day unit from datetime64, and
>> adding a business-day API would be a good approach to support all the
>> things that are needed?
>
> That sounds like a good idea to me -- and pe
On Mon, Jun 6, 2011 at 1:33 PM, Christopher Barker wrote:
> Mark Wiebe wrote:
> > I'm wondering if removing the business-day unit from datetime64, and
> > adding a business-day API would be a good approach to support all the
> > things that are needed?
>
> That sounds like a good idea to me -- and
Mark Wiebe wrote:
> I'm wondering if removing the business-day unit from datetime64, and
> adding a business-day API would be a good approach to support all the
> things that are needed?
That sounds like a good idea to me -- and perhaps it could be a general
Calendar Functions API, to handle ot
On Mon, Jun 6, 2011 at 4:43 AM, Wes McKinney wrote:
> On Mon, Jun 6, 2011 at 8:16 AM, Mark Dickinson
> wrote:
> > On Thu, Jun 2, 2011 at 5:42 PM, Mark Wiebe wrote:
> >> Leap years are easy compared with leap seconds. Leap seconds involve a
> >> hardcoded table of particular leap-seconds that ar
On Mon, Jun 6, 2011 at 2:16 AM, Mark Dickinson wrote:
> On Thu, Jun 2, 2011 at 5:42 PM, Mark Wiebe wrote:
> > Leap years are easy compared with leap seconds. Leap seconds involve a
> > hardcoded table of particular leap-seconds that are added or subtracted,
> and
> > are specified roughly 6 month
On Mon, Jun 6, 2011 at 8:16 AM, Mark Dickinson wrote:
> On Thu, Jun 2, 2011 at 5:42 PM, Mark Wiebe wrote:
>> Leap years are easy compared with leap seconds. Leap seconds involve a
>> hardcoded table of particular leap-seconds that are added or subtracted, and
>> are specified roughly 6 months in
On Thu, Jun 2, 2011 at 5:42 PM, Mark Wiebe wrote:
> Leap years are easy compared with leap seconds. Leap seconds involve a
> hardcoded table of particular leap-seconds that are added or subtracted, and
> are specified roughly 6 months in advance of when they happen by
> the International Earth Rot
On 6/2/11 12:57 PM, Robert Kern wrote:
>>> Anyhow, years and months are simple enough.
>>
>> no, they are not -- they are fundamentally different than hours, days, etc.
>
> That doesn't make them *difficult*.
I won't comment on how difficult it is -- I'm not writing the code. My
core point is tha
On Thu, Jun 2, 2011 at 13:44, Christopher Barker wrote:
> Pierre GM wrote:
>> Anyhow, years and months are simple enough.
>
> no, they are not -- they are fundamentally different than hours, days, etc.
That doesn't make them *difficult*. It's tricky to convert between
months and hours, yes, but
Mark Wiebe wrote:
> It is possible to implement the system so that if you don't use Y/M/B,
> things work out unambiguously, but if you do use them you get a behavior
> that's a little weird, but with rules to eliminate the calendar-created
> ambiguities.
yes, but everyone wants different rule
On Thu, Jun 2, 2011 at 12:44 PM, Christopher Barker
wrote:
> Pierre GM wrote:
> >> I also don't think that units like "month", "year", "business day"
> >> should be allowed -- it just adds confusion. It's not a killer if they
> >> are defined in the spec:
> >
> > -1 In scikits.timeseries, not only
On Thu, Jun 2, 2011 at 1:09 PM, Pierre GM wrote:
>
>
> > """
> > A representation is also supported such that the stored date-time
> > integer can encode both the number of a particular unit as well as a
> > number of sequential events tracked for each unit.
> > """
> >
> > I'm not sure I unde
Pierre GM wrote:
>> I also don't think that units like "month", "year", "business day"
>> should be allowed -- it just adds confusion. It's not a killer if they
>> are defined in the spec:
>
> -1 In scikits.timeseries, not only did we have years,months and
> business days, but we also had weeks,
Benjamin Root wrote:
> Just throwing it out there. Back in my Perl days (I do not often admit
> this), the one module that I thought was really well done was the
> datetime module. Now, I am not saying that we would need to support the
> calendar from Lord of the Rings or anything like that, b
On Jun 2, 2011, at 6:57 PM, Christopher Barker wrote:
> Mark Wiebe wrote:
>> I'm following what I understand the NEP to mean for combining dates and
>> deltas of different units. This means for timedeltas, the metadata
>> becomes more precise, in particular it becomes the GCD of the input
>> m
On Thu, Jun 2, 2011 at 10:57 AM, Christopher Barker
wrote:
> Mark Wiebe wrote:
> > I'm following what I understand the NEP to mean for combining dates and
> > deltas of different units. This means for timedeltas, the metadata
> > becomes more precise, in particular it becomes the GCD of the input
On Thu, Jun 2, 2011 at 12:17 PM, Benjamin Root wrote:
>
>
Just throwing it out there. Back in my Perl days (I do not often admit
> this), the one module that I thought was really well done was the datetime
> module. Now, I am not saying that we would need to support the calendar
> from Lord of
On Thu, Jun 2, 2011 at 11:42 AM, Mark Wiebe wrote:
> On Thu, Jun 2, 2011 at 11:22 AM, Christopher Barker > wrote:
>
>> Charles R Harris wrote:
>> > Good support for units and delta times is very useful, but
>> > parsing dates and times and handling timezones, daylight savings, leap
>> > seconds
On Thu, Jun 2, 2011 at 11:57 AM, Christopher Barker
wrote:
> Mark Wiebe wrote:
> > I'm following what I understand the NEP to mean for combining dates and
> > deltas of different units. This means for timedeltas, the metadata
> > becomes more precise, in particular it becomes the GCD of the input
Mark Wiebe wrote:
> I'm following what I understand the NEP to mean for combining dates and
> deltas of different units. This means for timedeltas, the metadata
> becomes more precise, in particular it becomes the GCD of the input
> metadata, and between timedelta and datetime the datetime alway
On Thu, Jun 2, 2011 at 11:22 AM, Christopher Barker
wrote:
> Charles R Harris wrote:
> > Good support for units and delta times is very useful, but
> > parsing dates and times and handling timezones, daylight savings, leap
> > seconds, business days, etc., is probably best served by addon package
On Thu, Jun 2, 2011 at 11:22, Christopher Barker wrote:
> Charles R Harris wrote:
>> * Leap seconds probably deserve a rigorous treatment, but having an
>> internal representation with leap-seconds overcomplicates otherwise very
>> simple and fast operations.
>
> could you explain more? I don't g
On Thu, Jun 2, 2011 at 10:36 AM, Pierre GM wrote:
>
> On Jun 1, 2011, at 11:16 PM, Mark Wiebe wrote:
>
> > On Wed, Jun 1, 2011 at 3:52 PM, Charles R Harris <
> charlesr.har...@gmail.com> wrote:
> >
> >
> > Just a quick comment, as this really needs more thought, but time is a
> bag of worms.
> >
Charles R Harris wrote:
> Good support for units and delta times is very useful, but
> parsing dates and times and handling timezones, daylight savings, leap
> seconds, business days, etc., is probably best served by addon packages
> specialized to an area of interest. Just my $.02
I agree here
On Jun 1, 2011, at 11:16 PM, Mark Wiebe wrote:
> On Wed, Jun 1, 2011 at 3:52 PM, Charles R Harris
> wrote:
>
>
> Just a quick comment, as this really needs more thought, but time is a bag of
> worms.
>
> Certainly a bag of worms, I agree.
Oh yes... Keep in mind that the inclusion of time i
On Thu, Jun 2, 2011 at 10:16 AM, Mark Wiebe wrote:
> On Thu, Jun 2, 2011 at 7:25 AM, Marc Shivers wrote:
>>
>> Regarding business days, for financial data, trading days are
>> determined by each exchange, so, in particular, there is no such thing
>> as a US Trading Calendar, only a NYSE Calendar,
On Thu, Jun 2, 2011 at 7:25 AM, Marc Shivers wrote:
> Regarding business days, for financial data, trading days are
> determined by each exchange, so, in particular, there is no such thing
> as a US Trading Calendar, only a NYSE Calendar, NYMEX Calendar, etc,
> etc... . I think it would be usefu
Regarding business days, for financial data, trading days are
determined by each exchange, so, in particular, there is no such thing
as a US Trading Calendar, only a NYSE Calendar, NYMEX Calendar, etc,
etc... . I think it would be useful to have functionality where you
could start with a weekday c
On Wed, Jun 1, 2011 at 10:29 PM, Charles R Harris
wrote:
>
>
> On Wed, Jun 1, 2011 at 3:16 PM, Mark Wiebe wrote:
>>
>> On Wed, Jun 1, 2011 at 3:52 PM, Charles R Harris
>> wrote:
>>>
>>>
>>> Just a quick comment, as this really needs more thought, but time is a
>>> bag of worms.
>>
>> Certainly
On Wed, Jun 1, 2011 at 3:16 PM, Mark Wiebe wrote:
> On Wed, Jun 1, 2011 at 3:52 PM, Charles R Harris <
> charlesr.har...@gmail.com> wrote:
>
>>
>>
>>
>> Just a quick comment, as this really needs more thought, but time is a bag
>> of worms.
>>
>
> Certainly a bag of worms, I agree.
>
>
>> Trying
On Wed, Jun 1, 2011 at 4:01 PM, Ralf Gommers wrote:
>
>
> On Wed, Jun 1, 2011 at 10:05 PM, Mark Wiebe wrote:
>
>> Hey all,
>>
>> So I'm doing a summer internship at Enthought, and the first thing they
>> asked me to look into is finishing the datetime type in numpy. It turns out
>> that the estim
On Wed, Jun 1, 2011 at 3:52 PM, Charles R Harris
wrote:
>
>
> Just a quick comment, as this really needs more thought, but time is a bag
> of worms.
>
Certainly a bag of worms, I agree.
> Trying to represent some standard -- say seconds at the solar system
> barycenter to account for general r
On Wed, Jun 1, 2011 at 11:04 PM, Charles R Harris wrote:
>
>
> On Wed, Jun 1, 2011 at 3:01 PM, Ralf Gommers
> wrote:
>
>>
>>
>> On Wed, Jun 1, 2011 at 10:05 PM, Mark Wiebe wrote:
>>
>>> Hey all,
>>>
>>> So I'm doing a summer internship at Enthought, and the first thing they
>>> asked me to look
On Wed, Jun 1, 2011 at 3:01 PM, Ralf Gommers wrote:
>
>
> On Wed, Jun 1, 2011 at 10:05 PM, Mark Wiebe wrote:
>
>> Hey all,
>>
>> So I'm doing a summer internship at Enthought, and the first thing they
>> asked me to look into is finishing the datetime type in numpy. It turns out
>> that the estim
On Wed, Jun 1, 2011 at 10:05 PM, Mark Wiebe wrote:
> Hey all,
>
> So I'm doing a summer internship at Enthought, and the first thing they
> asked me to look into is finishing the datetime type in numpy. It turns out
> that the estimates of how complete the type was weren't accurate, and to
> supp
On Wed, Jun 1, 2011 at 2:05 PM, Mark Wiebe wrote:
> Hey all,
>
> So I'm doing a summer internship at Enthought, and the first thing they
> asked me to look into is finishing the datetime type in numpy. It turns out
> that the estimates of how complete the type was weren't accurate, and to
> suppo
Hey all,
So I'm doing a summer internship at Enthought, and the first thing they
asked me to look into is finishing the datetime type in numpy. It turns out
that the estimates of how complete the type was weren't accurate, and to
support what the NEP describes required generalizing the ufunc type
70 matches
Mail list logo
