> @comaniac Having given this some thought, I think it's reasonable to support
> both approaches. I didn't want to include full logs because I was hoping to
> also be able to use config library to distribute tuned configs, however it
> should be fine to just 'export' a config library with only optimal configs.
>
Got your point, altough I think you can always pick the best config before
distribution like the current AutoTVM use case. Current AutoTVM log all configs
to a JSON file, and if a user only wants to keep the best one, she uses
`autotvm.record.pick_best` to generate another small JSON that only contains
the best config for each layer (task).
> In that case, I propose the following. Have each auto-tuning session create a
> new 'job'. This job will have an entry in a JSON file ('job index')
> containing at least the target string, start/finish time of the job and a
> path to the history file generated. Optionally we permit some arbitrary JSON
> to describe the platform in more detail. By default, we delete the history
> file when a job completes (but keep the job entry in the index), however a
> flag can be passed to retain the history.
>
If I undertand correctly, you are going to add tuning process metadata to the
JSON file in addition to the configs, like the example code snippet you
proposed in the very beginning of this RFC. Since you propose to use config
library as the "database" to log all configs (the argument of `log_to_file`
callback), you have to make sure the current `history.json` is still always
available with explored configs whenever the task is interrupted. Maybe we can
store two JSON files (e.g., task.json and history.json) to make a clear
separation?
Another suggestion is naming. `ConfigLibrary` seems not accurate in this case.
To me, it's more like a `TaskContext` or `TaskSession`. Also we should use task
instead of job to be consistent.
> Now if a task needs to be resume, first a simple check can be done to see if
> the existing optimal config has already been tuned with sufficiently many
> trials (and with the right tuner/platform). If so, skip, otherwise search the
> job index to see if any history files qualify to restart the tuning. In that
> case, we can use your proposal.
Yeah I think this part is relatively clear.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/dmlc/tvm/issues/4150#issuecomment-545097314
