Some LLAMA benchmark results

LLAMA results

All results were produced by using the cross-validation splits in the repository with 10 folds and 1 repetitions.
The best values within a type (i.e., baseline (except for vbs), classif, regr and cluster) and performance measure (i.e., Percentage solved, PAR10, MCP) are colored green. Furthermore, the three best values over all groups within a performance measure are colored pink, the absolute best one is red.

The performance is measured in three different ways.

Percentage solved records the percentage of problem instances in the data set for which the selector selected an algorithm that was able to solve it with runstatus "ok" and the algorithm time plus the feature computation time was at most the timeout.
The penalized average runtime score (PAR10) measures the time required to run on all problem instances. If an instance was solved within the timeout by the algorithm the selector chose, the actual runtime is taken. If a timeout occurred, the timeout value was multiplied by 10.
The misclassification penalty (mcp) measures the additional time required to run on all problems if sub-optimal algorithms were used. That is, if an algorithm is run on a problem instance that is not the best, a performance loss is incurred. There are no additional penalties or factors for timeouts. The virtual best solver always has a misclassification penalty of zero.

algo	model	succ	par10	mcp
baseline	vbs	1.000	227.605	0.000
baseline	singleBest	0.787	7945.039	831.666
baseline	singleBestByPar	0.812	7002.907	688.774
baseline	singleBestBySuccesses	0.812	7002.907	688.774
classif	meta/AdaBoostM1	0.825	6527.829	644.058
classif	bayes/BayesNet	0.827	6405.856	583.564
classif	lazy/IBk	0.837	6096.055	581.163
classif	rules/OneR	0.850	5633.623	549.093
classif	trees/RandomTree	0.824	6565.724	620.472
classif	trees/J48	0.846	5764.771	557.280
classif	rules/JRip	0.835	6216.069	639.698
classif	classif.ctree	0.867	5017.059	485.849
classif	classif.ksvm	0.843	5885.958	555.507
classif	classif.naiveBayes	0.774	8275.600	731.866
classif	classif.randomForest	0.860	5293.406	516.276
classif	classif.rpart	0.873	4832.339	485.569
regr	regr.lm	0.843	5885.054	554.603
regr	regr.rpart	0.850	5645.921	561.391
regr	regr.randomForest	0.844	5815.493	546.522
regr	regr.earth	0.863	5136.973	482.803
cluster	EM	0.787	7945.644	832.271
cluster	FarthestFirst	0.824	6601.998	656.746
cluster	SimpleKMeans	0.825	6524.254	640.482

The following default feature steps were used for model building:

all

Number of presolved instances: 0

The cost for using the feature steps (adapted for presolving) is: 0 or on average: NA

The feature steps correspond to the following 16 / 16 features:

stacks, tiers, stack.tier.ratio, container.density, empty.stack.pct,
overstowing.stack.pct, group.same.min, group.same.max, group.same.mean, group.same.stdev,
top.good.min, top.good.max, top.good.mean, top.good.stdev, overstowage.pct,
bflb,