<spanclass="sig-prename descclassname"><spanclass="pre">quapy.error.</span></span><spanclass="sig-name descname"><spanclass="pre">absolute_error</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">prevs</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">prevs_hat</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.error.absolute_error"title="Permalink to this definition">¶</a></dt>
<dd><dlclass="simple">
<dt>Computes the absolute error between the two prevalence vectors.</dt><dd><p>Absolute error between two prevalence vectors <spanclass="math notranslate nohighlight">\(p\)</span> and <spanclass="math notranslate nohighlight">\(\hat{p}\)</span> is computed as
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.error.</span></span><spanclass="sig-name descname"><spanclass="pre">acc_error</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">y_true</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">y_pred</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.error.acc_error"title="Permalink to this definition">¶</a></dt>
<dd><p>Computes the error in terms of 1-accuracy. The accuracy is computed as <spanclass="math notranslate nohighlight">\(\frac{tp+tn}{tp+fp+fn+tn}\)</span>, with
<cite>tp</cite>, <cite>fp</cite>, <cite>fn</cite>, and <cite>tn</cite> standing for true positives, false positives, false negatives, and true negatives,
respectively</p>
<dlclass="field-list simple">
<dtclass="field-odd">Parameters</dt>
<ddclass="field-odd"><ulclass="simple">
<li><p><strong>y_true</strong>– array-like of true labels</p></li>
<li><p><strong>y_pred</strong>– array-like of predicted labels</p></li>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.error.</span></span><spanclass="sig-name descname"><spanclass="pre">acce</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">y_true</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">y_pred</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.error.acce"title="Permalink to this definition">¶</a></dt>
<dd><p>Computes the error in terms of 1-accuracy. The accuracy is computed as <spanclass="math notranslate nohighlight">\(\frac{tp+tn}{tp+fp+fn+tn}\)</span>, with
<cite>tp</cite>, <cite>fp</cite>, <cite>fn</cite>, and <cite>tn</cite> standing for true positives, false positives, false negatives, and true negatives,
respectively</p>
<dlclass="field-list simple">
<dtclass="field-odd">Parameters</dt>
<ddclass="field-odd"><ulclass="simple">
<li><p><strong>y_true</strong>– array-like of true labels</p></li>
<li><p><strong>y_pred</strong>– array-like of predicted labels</p></li>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.error.</span></span><spanclass="sig-name descname"><spanclass="pre">ae</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">prevs</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">prevs_hat</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.error.ae"title="Permalink to this definition">¶</a></dt>
<dd><dlclass="simple">
<dt>Computes the absolute error between the two prevalence vectors.</dt><dd><p>Absolute error between two prevalence vectors <spanclass="math notranslate nohighlight">\(p\)</span> and <spanclass="math notranslate nohighlight">\(\hat{p}\)</span> is computed as
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.error.</span></span><spanclass="sig-name descname"><spanclass="pre">f1_error</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">y_true</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">y_pred</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.error.f1_error"title="Permalink to this definition">¶</a></dt>
<dd><p>F1 error: simply computes the error in terms of macro <spanclass="math notranslate nohighlight">\(F_1\)</span>, i.e., <spanclass="math notranslate nohighlight">\(1-F_1^M\)</span>,
where <spanclass="math notranslate nohighlight">\(F_1\)</span> is the harmonic mean of precision and recall, defined as <spanclass="math notranslate nohighlight">\(\frac{2tp}{2tp+fp+fn}\)</span>,
with <cite>tp</cite>, <cite>fp</cite>, and <cite>fn</cite> standing for true positives, false positives, and false negatives, respectively.
<cite>Macro</cite> averaging means the <spanclass="math notranslate nohighlight">\(F_1\)</span> is computed for each category independently, and then averaged.</p>
<dlclass="field-list simple">
<dtclass="field-odd">Parameters</dt>
<ddclass="field-odd"><ulclass="simple">
<li><p><strong>y_true</strong>– array-like of true labels</p></li>
<li><p><strong>y_pred</strong>– array-like of predicted labels</p></li>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.error.</span></span><spanclass="sig-name descname"><spanclass="pre">f1e</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">y_true</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">y_pred</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.error.f1e"title="Permalink to this definition">¶</a></dt>
<dd><p>F1 error: simply computes the error in terms of macro <spanclass="math notranslate nohighlight">\(F_1\)</span>, i.e., <spanclass="math notranslate nohighlight">\(1-F_1^M\)</span>,
where <spanclass="math notranslate nohighlight">\(F_1\)</span> is the harmonic mean of precision and recall, defined as <spanclass="math notranslate nohighlight">\(\frac{2tp}{2tp+fp+fn}\)</span>,
with <cite>tp</cite>, <cite>fp</cite>, and <cite>fn</cite> standing for true positives, false positives, and false negatives, respectively.
<cite>Macro</cite> averaging means the <spanclass="math notranslate nohighlight">\(F_1\)</span> is computed for each category independently, and then averaged.</p>
<dlclass="field-list simple">
<dtclass="field-odd">Parameters</dt>
<ddclass="field-odd"><ulclass="simple">
<li><p><strong>y_true</strong>– array-like of true labels</p></li>
<li><p><strong>y_pred</strong>– array-like of predicted labels</p></li>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.error.</span></span><spanclass="sig-name descname"><spanclass="pre">from_name</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">err_name</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.error.from_name"title="Permalink to this definition">¶</a></dt>
<dd><p>Gets an error function from its name. E.g., <cite>from_name(“mae”)</cite> will return function <aclass="reference internal"href="#quapy.error.mae"title="quapy.error.mae"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">quapy.error.mae()</span></code></a></p>
<dlclass="field-list simple">
<dtclass="field-odd">Parameters</dt>
<ddclass="field-odd"><p><strong>err_name</strong>– string, the error name</p>
</dd>
<dtclass="field-even">Returns</dt>
<ddclass="field-even"><p>a callable implementing the requested error</p>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.error.</span></span><spanclass="sig-name descname"><spanclass="pre">kld</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">p</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">p_hat</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">eps</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">None</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.error.kld"title="Permalink to this definition">¶</a></dt>
<dt>Computes the Kullback-Leibler divergence between the two prevalence distributions.</dt><dd><p>Kullback-Leibler divergence between two prevalence distributions <spanclass="math notranslate nohighlight">\(p\)</span> and <spanclass="math notranslate nohighlight">\(\hat{p}\)</span> is computed as
<spanclass="math notranslate nohighlight">\(KLD(p,\hat{p})=D_{KL}(p||\hat{p})=\sum_{y\in \mathcal{Y}} p(y)\log\frac{p(y)}{\hat{p}(y)}\)</span>, where
<spanclass="math notranslate nohighlight">\(\mathcal{Y}\)</span> are the classes of interest.
The distributions are smoothed using the <cite>eps</cite> factor (see <aclass="reference internal"href="#quapy.error.smooth"title="quapy.error.smooth"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">quapy.error.smooth()</span></code></a>).</p>
</dd>
</dl>
<dlclass="field-list simple">
<dtclass="field-odd">Parameters</dt>
<ddclass="field-odd"><ulclass="simple">
<li><p><strong>prevs</strong>– array-like of shape <cite>(n_classes,)</cite> with the true prevalence values</p></li>
<li><p><strong>prevs_hat</strong>– array-like of shape <cite>(n_classes,)</cite> with the predicted prevalence values</p></li>
<li><p><strong>eps</strong>– smoothing factor. KLD is not defined in cases in which the distributions contain zeros; <cite>eps</cite>
is typically set to be <spanclass="math notranslate nohighlight">\(\frac{1}{2T}\)</span>, with <spanclass="math notranslate nohighlight">\(T\)</span> the sample size. If <cite>eps=None</cite>, the sample size
will be taken from the environment variable <cite>SAMPLE_SIZE</cite> (which has thus to be set beforehand).</p></li>
</ul>
</dd>
<dtclass="field-even">Returns</dt>
<ddclass="field-even"><p>Kullback-Leibler divergence between the two distributions</p>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.error.</span></span><spanclass="sig-name descname"><spanclass="pre">mae</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">prevs</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">prevs_hat</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.error.mae"title="Permalink to this definition">¶</a></dt>
<dd><p>Computes the mean absolute error (see <aclass="reference internal"href="#quapy.error.ae"title="quapy.error.ae"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">quapy.error.ae()</span></code></a>) across the sample pairs.</p>
<dlclass="field-list simple">
<dtclass="field-odd">Parameters</dt>
<ddclass="field-odd"><ulclass="simple">
<li><p><strong>prevs</strong>– array-like of shape <cite>(n_samples, n_classes,)</cite> with the true prevalence values</p></li>
<li><p><strong>prevs_hat</strong>– array-like of shape <cite>(n_samples, n_classes,)</cite> with the predicted prevalence values</p></li>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.error.</span></span><spanclass="sig-name descname"><spanclass="pre">mean_absolute_error</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">prevs</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">prevs_hat</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.error.mean_absolute_error"title="Permalink to this definition">¶</a></dt>
<dd><p>Computes the mean absolute error (see <aclass="reference internal"href="#quapy.error.ae"title="quapy.error.ae"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">quapy.error.ae()</span></code></a>) across the sample pairs.</p>
<dlclass="field-list simple">
<dtclass="field-odd">Parameters</dt>
<ddclass="field-odd"><ulclass="simple">
<li><p><strong>prevs</strong>– array-like of shape <cite>(n_samples, n_classes,)</cite> with the true prevalence values</p></li>
<li><p><strong>prevs_hat</strong>– array-like of shape <cite>(n_samples, n_classes,)</cite> with the predicted prevalence values</p></li>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.error.</span></span><spanclass="sig-name descname"><spanclass="pre">mean_relative_absolute_error</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">p</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">p_hat</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">eps</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">None</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.error.mean_relative_absolute_error"title="Permalink to this definition">¶</a></dt>
<dd><p>Computes the mean relative absolute error (see <aclass="reference internal"href="#quapy.error.rae"title="quapy.error.rae"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">quapy.error.rae()</span></code></a>) across the sample pairs.
The distributions are smoothed using the <cite>eps</cite> factor (see <aclass="reference internal"href="#quapy.error.smooth"title="quapy.error.smooth"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">quapy.error.smooth()</span></code></a>).</p>
<dlclass="field-list simple">
<dtclass="field-odd">Parameters</dt>
<ddclass="field-odd"><ulclass="simple">
<li><p><strong>prevs</strong>– array-like of shape <cite>(n_samples, n_classes,)</cite> with the true prevalence values</p></li>
<li><p><strong>prevs_hat</strong>– array-like of shape <cite>(n_samples, n_classes,)</cite> with the predicted prevalence values</p></li>
<li><p><strong>eps</strong>– smoothing factor. <cite>mrae</cite> is not defined in cases in which the true distribution contains zeros; <cite>eps</cite>
is typically set to be <spanclass="math notranslate nohighlight">\(\frac{1}{2T}\)</span>, with <spanclass="math notranslate nohighlight">\(T\)</span> the sample size. If <cite>eps=None</cite>, the sample size
will be taken from the environment variable <cite>SAMPLE_SIZE</cite> (which has thus to be set beforehand).</p></li>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.error.</span></span><spanclass="sig-name descname"><spanclass="pre">mkld</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">prevs</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">prevs_hat</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">eps</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">None</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.error.mkld"title="Permalink to this definition">¶</a></dt>
<dd><p>Computes the mean Kullback-Leibler divergence (see <aclass="reference internal"href="#quapy.error.kld"title="quapy.error.kld"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">quapy.error.kld()</span></code></a>) across the sample pairs.
The distributions are smoothed using the <cite>eps</cite> factor (see <aclass="reference internal"href="#quapy.error.smooth"title="quapy.error.smooth"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">quapy.error.smooth()</span></code></a>).</p>
<dlclass="field-list simple">
<dtclass="field-odd">Parameters</dt>
<ddclass="field-odd"><ulclass="simple">
<li><p><strong>prevs</strong>– array-like of shape <cite>(n_samples, n_classes,)</cite> with the true prevalence values</p></li>
<li><p><strong>prevs_hat</strong>– array-like of shape <cite>(n_samples, n_classes,)</cite> with the predicted prevalence values</p></li>
<li><p><strong>eps</strong>– smoothing factor. KLD is not defined in cases in which the distributions contain zeros; <cite>eps</cite>
is typically set to be <spanclass="math notranslate nohighlight">\(\frac{1}{2T}\)</span>, with <spanclass="math notranslate nohighlight">\(T\)</span> the sample size. If <cite>eps=None</cite>, the sample size
will be taken from the environment variable <cite>SAMPLE_SIZE</cite> (which has thus to be set beforehand).</p></li>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.error.</span></span><spanclass="sig-name descname"><spanclass="pre">mnkld</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">prevs</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">prevs_hat</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">eps</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">None</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.error.mnkld"title="Permalink to this definition">¶</a></dt>
<dd><p>Computes the mean Normalized Kullback-Leibler divergence (see <aclass="reference internal"href="#quapy.error.nkld"title="quapy.error.nkld"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">quapy.error.nkld()</span></code></a>) across the sample pairs.
The distributions are smoothed using the <cite>eps</cite> factor (see <aclass="reference internal"href="#quapy.error.smooth"title="quapy.error.smooth"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">quapy.error.smooth()</span></code></a>).</p>
<dlclass="field-list simple">
<dtclass="field-odd">Parameters</dt>
<ddclass="field-odd"><ulclass="simple">
<li><p><strong>prevs</strong>– array-like of shape <cite>(n_samples, n_classes,)</cite> with the true prevalence values</p></li>
<li><p><strong>prevs_hat</strong>– array-like of shape <cite>(n_samples, n_classes,)</cite> with the predicted prevalence values</p></li>
<li><p><strong>eps</strong>– smoothing factor. NKLD is not defined in cases in which the distributions contain zeros; <cite>eps</cite>
is typically set to be <spanclass="math notranslate nohighlight">\(\frac{1}{2T}\)</span>, with <spanclass="math notranslate nohighlight">\(T\)</span> the sample size. If <cite>eps=None</cite>, the sample size
will be taken from the environment variable <cite>SAMPLE_SIZE</cite> (which has thus to be set beforehand).</p></li>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.error.</span></span><spanclass="sig-name descname"><spanclass="pre">mrae</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">p</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">p_hat</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">eps</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">None</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.error.mrae"title="Permalink to this definition">¶</a></dt>
<dd><p>Computes the mean relative absolute error (see <aclass="reference internal"href="#quapy.error.rae"title="quapy.error.rae"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">quapy.error.rae()</span></code></a>) across the sample pairs.
The distributions are smoothed using the <cite>eps</cite> factor (see <aclass="reference internal"href="#quapy.error.smooth"title="quapy.error.smooth"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">quapy.error.smooth()</span></code></a>).</p>
<dlclass="field-list simple">
<dtclass="field-odd">Parameters</dt>
<ddclass="field-odd"><ulclass="simple">
<li><p><strong>prevs</strong>– array-like of shape <cite>(n_samples, n_classes,)</cite> with the true prevalence values</p></li>
<li><p><strong>prevs_hat</strong>– array-like of shape <cite>(n_samples, n_classes,)</cite> with the predicted prevalence values</p></li>
<li><p><strong>eps</strong>– smoothing factor. <cite>mrae</cite> is not defined in cases in which the true distribution contains zeros; <cite>eps</cite>
is typically set to be <spanclass="math notranslate nohighlight">\(\frac{1}{2T}\)</span>, with <spanclass="math notranslate nohighlight">\(T\)</span> the sample size. If <cite>eps=None</cite>, the sample size
will be taken from the environment variable <cite>SAMPLE_SIZE</cite> (which has thus to be set beforehand).</p></li>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.error.</span></span><spanclass="sig-name descname"><spanclass="pre">mse</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">prevs</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">prevs_hat</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.error.mse"title="Permalink to this definition">¶</a></dt>
<dd><p>Computes the mean squared error (see <aclass="reference internal"href="#quapy.error.se"title="quapy.error.se"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">quapy.error.se()</span></code></a>) across the sample pairs.</p>
<dlclass="field-list simple">
<dtclass="field-odd">Parameters</dt>
<ddclass="field-odd"><ulclass="simple">
<li><p><strong>prevs</strong>– array-like of shape <cite>(n_samples, n_classes,)</cite> with the true prevalence values</p></li>
<li><p><strong>prevs_hat</strong>– array-like of shape <cite>(n_samples, n_classes,)</cite> with the predicted prevalence values</p></li>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.error.</span></span><spanclass="sig-name descname"><spanclass="pre">nkld</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">p</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">p_hat</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">eps</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">None</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.error.nkld"title="Permalink to this definition">¶</a></dt>
<dt>Computes the Normalized Kullback-Leibler divergence between the two prevalence distributions.</dt><dd><p>Normalized Kullback-Leibler divergence between two prevalence distributions <spanclass="math notranslate nohighlight">\(p\)</span> and <spanclass="math notranslate nohighlight">\(\hat{p}\)</span>
is computed as <spanclass="math notranslate nohighlight">\(NKLD(p,\hat{p}) = 2\frac{e^{KLD(p,\hat{p})}}{e^{KLD(p,\hat{p})}+1}-1\)</span>, where
<spanclass="math notranslate nohighlight">\(\mathcal{Y}\)</span> are the classes of interest.
The distributions are smoothed using the <cite>eps</cite> factor (see <aclass="reference internal"href="#quapy.error.smooth"title="quapy.error.smooth"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">quapy.error.smooth()</span></code></a>).</p>
</dd>
</dl>
<dlclass="field-list simple">
<dtclass="field-odd">Parameters</dt>
<ddclass="field-odd"><ulclass="simple">
<li><p><strong>prevs</strong>– array-like of shape <cite>(n_classes,)</cite> with the true prevalence values</p></li>
<li><p><strong>prevs_hat</strong>– array-like of shape <cite>(n_classes,)</cite> with the predicted prevalence values</p></li>
<li><p><strong>eps</strong>– smoothing factor. NKLD is not defined in cases in which the distributions contain zeros; <cite>eps</cite>
is typically set to be <spanclass="math notranslate nohighlight">\(\frac{1}{2T}\)</span>, with <spanclass="math notranslate nohighlight">\(T\)</span> the sample size. If <cite>eps=None</cite>, the sample size
will be taken from the environment variable <cite>SAMPLE_SIZE</cite> (which has thus to be set beforehand).</p></li>
</ul>
</dd>
<dtclass="field-even">Returns</dt>
<ddclass="field-even"><p>Normalized Kullback-Leibler divergence between the two distributions</p>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.error.</span></span><spanclass="sig-name descname"><spanclass="pre">rae</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">p</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">p_hat</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">eps</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">None</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.error.rae"title="Permalink to this definition">¶</a></dt>
<dt>Computes the absolute relative error between the two prevalence vectors.</dt><dd><p>Relative absolute error between two prevalence vectors <spanclass="math notranslate nohighlight">\(p\)</span> and <spanclass="math notranslate nohighlight">\(\hat{p}\)</span> is computed as
where <spanclass="math notranslate nohighlight">\(\mathcal{Y}\)</span> are the classes of interest.
The distributions are smoothed using the <cite>eps</cite> factor (see <aclass="reference internal"href="#quapy.error.smooth"title="quapy.error.smooth"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">quapy.error.smooth()</span></code></a>).</p>
</dd>
</dl>
<dlclass="field-list simple">
<dtclass="field-odd">Parameters</dt>
<ddclass="field-odd"><ulclass="simple">
<li><p><strong>prevs</strong>– array-like of shape <cite>(n_classes,)</cite> with the true prevalence values</p></li>
<li><p><strong>prevs_hat</strong>– array-like of shape <cite>(n_classes,)</cite> with the predicted prevalence values</p></li>
<li><p><strong>eps</strong>– smoothing factor. <cite>rae</cite> is not defined in cases in which the true distribution contains zeros; <cite>eps</cite>
is typically set to be <spanclass="math notranslate nohighlight">\(\frac{1}{2T}\)</span>, with <spanclass="math notranslate nohighlight">\(T\)</span> the sample size. If <cite>eps=None</cite>, the sample size
will be taken from the environment variable <cite>SAMPLE_SIZE</cite> (which has thus to be set beforehand).</p></li>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.error.</span></span><spanclass="sig-name descname"><spanclass="pre">relative_absolute_error</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">p</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">p_hat</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">eps</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">None</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.error.relative_absolute_error"title="Permalink to this definition">¶</a></dt>
<dt>Computes the absolute relative error between the two prevalence vectors.</dt><dd><p>Relative absolute error between two prevalence vectors <spanclass="math notranslate nohighlight">\(p\)</span> and <spanclass="math notranslate nohighlight">\(\hat{p}\)</span> is computed as
where <spanclass="math notranslate nohighlight">\(\mathcal{Y}\)</span> are the classes of interest.
The distributions are smoothed using the <cite>eps</cite> factor (see <aclass="reference internal"href="#quapy.error.smooth"title="quapy.error.smooth"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">quapy.error.smooth()</span></code></a>).</p>
</dd>
</dl>
<dlclass="field-list simple">
<dtclass="field-odd">Parameters</dt>
<ddclass="field-odd"><ulclass="simple">
<li><p><strong>prevs</strong>– array-like of shape <cite>(n_classes,)</cite> with the true prevalence values</p></li>
<li><p><strong>prevs_hat</strong>– array-like of shape <cite>(n_classes,)</cite> with the predicted prevalence values</p></li>
<li><p><strong>eps</strong>– smoothing factor. <cite>rae</cite> is not defined in cases in which the true distribution contains zeros; <cite>eps</cite>
is typically set to be <spanclass="math notranslate nohighlight">\(\frac{1}{2T}\)</span>, with <spanclass="math notranslate nohighlight">\(T\)</span> the sample size. If <cite>eps=None</cite>, the sample size
will be taken from the environment variable <cite>SAMPLE_SIZE</cite> (which has thus to be set beforehand).</p></li>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.error.</span></span><spanclass="sig-name descname"><spanclass="pre">se</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">p</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">p_hat</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.error.se"title="Permalink to this definition">¶</a></dt>
<dt>Computes the squared error between the two prevalence vectors.</dt><dd><p>Squared error between two prevalence vectors <spanclass="math notranslate nohighlight">\(p\)</span> and <spanclass="math notranslate nohighlight">\(\hat{p}\)</span> is computed as
<spanclass="math notranslate nohighlight">\(SE(p,\hat{p})=\frac{1}{|\mathcal{Y}|}\sum_{y\in \mathcal{Y}}(\hat{p}(y)-p(y))^2\)</span>, where
<spanclass="math notranslate nohighlight">\(\mathcal{Y}\)</span> are the classes of interest.</p>
</dd>
</dl>
<dlclass="field-list simple">
<dtclass="field-odd">Parameters</dt>
<ddclass="field-odd"><ulclass="simple">
<li><p><strong>prevs</strong>– array-like of shape <cite>(n_classes,)</cite> with the true prevalence values</p></li>
<li><p><strong>prevs_hat</strong>– array-like of shape <cite>(n_classes,)</cite> with the predicted prevalence values</p></li>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.error.</span></span><spanclass="sig-name descname"><spanclass="pre">smooth</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">prevs</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">eps</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.error.smooth"title="Permalink to this definition">¶</a></dt>
<dd><p>Smooths a prevalence distribution with <spanclass="math notranslate nohighlight">\(\epsilon\)</span> (<cite>eps</cite>) as:
<spanid="quapy-evaluation-module"></span><h2>quapy.evaluation module<aclass="headerlink"href="#module-quapy.evaluation"title="Permalink to this headline">¶</a></h2>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.evaluation.</span></span><spanclass="sig-name descname"><spanclass="pre">artificial_prevalence_protocol</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">model</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="n"><aclass="reference internal"href="quapy.method.html#quapy.method.base.BaseQuantifier"title="quapy.method.base.BaseQuantifier"><spanclass="pre">quapy.method.base.BaseQuantifier</span></a></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">test</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="n"><aclass="reference internal"href="quapy.data.html#quapy.data.base.LabelledCollection"title="quapy.data.base.LabelledCollection"><spanclass="pre">quapy.data.base.LabelledCollection</span></a></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">sample_size</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">n_prevpoints</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">101</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">repeats</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">1</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">eval_budget</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="n"><spanclass="pre">Optional</span><spanclass="p"><spanclass="pre">[</span></span><spanclass="pre">int</span><spanclass="p"><spanclass="pre">]</span></span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">None</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">n_jobs</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">1</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">random_seed</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">42</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">error_metric</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="n"><spanclass="pre">Union</span><spanclass="p"><spanclass="pre">[</span></span><spanclass="pre">str</span><spanclass="p"><spanclass="pre">,</span></span><spanclass="pre">Callable</span><spanclass="p"><spanclass="pre">]</span></span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">'mae'</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">verbose</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">False</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.evaluation.artificial_prevalence_protocol"title="Permalink to this definition">¶</a></dt>
<dd><p>Generates samples according to the Artificial Prevalence Protocol (APP).
The APP consists of exploring a grid of prevalence values containing <cite>n_prevalences</cite> points (e.g.,
[0, 0.05, 0.1, 0.15, …, 1], if <cite>n_prevalences=21</cite>), and generating all valid combinations of
prevalence values for all classes (e.g., for 3 classes, samples with [0, 0, 1], [0, 0.05, 0.95], …,
[1, 0, 0] prevalence values of size <cite>sample_size</cite> will be considered). The number of samples for each valid
combination of prevalence values is indicated by <cite>repeats</cite>.</p>
<dlclass="field-list simple">
<dtclass="field-odd">Parameters</dt>
<ddclass="field-odd"><ulclass="simple">
<li><p><strong>model</strong>– the model in charge of generating the class prevalence estimations</p></li>
<li><p><strong>test</strong>– the test set on which to perform APP</p></li>
<li><p><strong>sample_size</strong>– integer, the size of the samples</p></li>
<li><p><strong>n_prevpoints</strong>– integer, the number of different prevalences to sample (or set to None if eval_budget
is specified; default 101, i.e., steps of 1%)</p></li>
<li><p><strong>repeats</strong>– integer, the number of repetitions for each prevalence (default 1)</p></li>
<li><p><strong>eval_budget</strong>– integer, if specified, sets a ceil on the number of evaluations to perform. For example, if
there are 3 classes, <cite>repeats=1</cite>, and <cite>eval_budget=20</cite>, then <cite>n_prevpoints</cite> will be set to 5, since this
will generate 15 different prevalence vectors ([0, 0, 1], [0, 0.25, 0.75], [0, 0.5, 0.5] … [1, 0, 0]) and
since setting <cite>n_prevpoints=6</cite> would produce more than 20 evaluations.</p></li>
<li><p><strong>n_jobs</strong>– integer, number of jobs to be run in parallel (default 1)</p></li>
<li><p><strong>random_seed</strong>– integer, allows to replicate the samplings. The seed is local to the method and does not affect
any other random process (default 42)</p></li>
<li><p><strong>error_metric</strong>– a string indicating the name of the error (as defined in <aclass="reference internal"href="#module-quapy.error"title="quapy.error"><codeclass="xref py py-mod docutils literal notranslate"><spanclass="pre">quapy.error</span></code></a>) or a
callable error function</p></li>
<li><p><strong>verbose</strong>– set to True (default False) for displaying some information on standard output</p></li>
</ul>
</dd>
<dtclass="field-even">Returns</dt>
<ddclass="field-even"><p>yields one sample at a time</p>
in which the rows correspond to different samples, and the columns inform of the true prevalence values,
the estimated prevalence values, and the score obtained by each of the evaluation measures indicated.</p>
<dlclass="field-list simple">
<dtclass="field-odd">Parameters</dt>
<ddclass="field-odd"><ulclass="simple">
<li><p><strong>model</strong>– the model in charge of generating the class prevalence estimations</p></li>
<li><p><strong>test</strong>– the test set on which to perform APP</p></li>
<li><p><strong>sample_size</strong>– integer, the size of the samples</p></li>
<li><p><strong>n_prevpoints</strong>– integer, the number of different prevalences to sample (or set to None if eval_budget
is specified; default 101, i.e., steps of 1%)</p></li>
<li><p><strong>repeats</strong>– integer, the number of repetitions for each prevalence (default 1)</p></li>
<li><p><strong>eval_budget</strong>– integer, if specified, sets a ceil on the number of evaluations to perform. For example, if
there are 3 classes, <cite>repeats=1</cite>, and <cite>eval_budget=20</cite>, then <cite>n_prevpoints</cite> will be set to 5, since this
will generate 15 different prevalence vectors ([0, 0, 1], [0, 0.25, 0.75], [0, 0.5, 0.5] … [1, 0, 0]) and
since setting <cite>n_prevpoints=6</cite> would produce more than 20 evaluations.</p></li>
<li><p><strong>n_jobs</strong>– integer, number of jobs to be run in parallel (default 1)</p></li>
<li><p><strong>random_seed</strong>– integer, allows to replicate the samplings. The seed is local to the method and does not affect
any other random process (default 42)</p></li>
<li><p><strong>error_metrics</strong>– a string indicating the name of the error (as defined in <aclass="reference internal"href="#module-quapy.error"title="quapy.error"><codeclass="xref py py-mod docutils literal notranslate"><spanclass="pre">quapy.error</span></code></a>) or a
callable error function; optionally, a list of strings or callables can be indicated, if the results
are to be evaluated with more than one error metric. Default is “mae”</p></li>
<li><p><strong>verbose</strong>– if True, shows a progress bar</p></li>
</ul>
</dd>
<dtclass="field-even">Returns</dt>
<ddclass="field-even"><p>pandas’ dataframe with rows corresponding to different samples, and with columns informing of the
true prevalence values, the estimated prevalence values, and the score obtained by each of the evaluation
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.evaluation.</span></span><spanclass="sig-name descname"><spanclass="pre">evaluate</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">model</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="n"><aclass="reference internal"href="quapy.method.html#quapy.method.base.BaseQuantifier"title="quapy.method.base.BaseQuantifier"><spanclass="pre">quapy.method.base.BaseQuantifier</span></a></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">test_samples</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="n"><spanclass="pre">Iterable</span><spanclass="p"><spanclass="pre">[</span></span><aclass="reference internal"href="quapy.data.html#quapy.data.base.LabelledCollection"title="quapy.data.base.LabelledCollection"><spanclass="pre">quapy.data.base.LabelledCollection</span></a><spanclass="p"><spanclass="pre">]</span></span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">error_metric</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="n"><spanclass="pre">Union</span><spanclass="p"><spanclass="pre">[</span></span><spanclass="pre">str</span><spanclass="p"><spanclass="pre">,</span></span><spanclass="pre">Callable</span><spanclass="p"><spanclass="pre">]</span></span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">n_jobs</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="n"><spanclass="pre">int</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">-</span><spanclass="pre">1</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.evaluation.evaluate"title="Permalink to this definition">¶</a></dt>
<dd><p>Evaluates a model on a sequence of test samples in terms of a given error metric.</p>
<dlclass="field-list simple">
<dtclass="field-odd">Parameters</dt>
<ddclass="field-odd"><ulclass="simple">
<li><p><strong>model</strong>– the model in charge of generating the class prevalence estimations</p></li>
<li><p><strong>test_samples</strong>– an iterable yielding one sample at a time</p></li>
<li><p><strong>error_metric</strong>– a string indicating the name of the error (as defined in <aclass="reference internal"href="#module-quapy.error"title="quapy.error"><codeclass="xref py py-mod docutils literal notranslate"><spanclass="pre">quapy.error</span></code></a>) or a
callable error function</p></li>
<li><p><strong>n_jobs</strong>– integer, number of jobs to be run in parallel (default 1)</p></li>
</ul>
</dd>
<dtclass="field-even">Returns</dt>
<ddclass="field-even"><p>the score obtained using <cite>error_metric</cite></p>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.evaluation.</span></span><spanclass="sig-name descname"><spanclass="pre">gen_prevalence_prediction</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">model</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="n"><aclass="reference internal"href="quapy.method.html#quapy.method.base.BaseQuantifier"title="quapy.method.base.BaseQuantifier"><spanclass="pre">quapy.method.base.BaseQuantifier</span></a></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">gen_fn</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="n"><spanclass="pre">Callable</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">eval_budget</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">None</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.evaluation.gen_prevalence_prediction"title="Permalink to this definition">¶</a></dt>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.evaluation.</span></span><spanclass="sig-name descname"><spanclass="pre">gen_prevalence_report</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">model</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="n"><aclass="reference internal"href="quapy.method.html#quapy.method.base.BaseQuantifier"title="quapy.method.base.BaseQuantifier"><spanclass="pre">quapy.method.base.BaseQuantifier</span></a></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">gen_fn</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="n"><spanclass="pre">Callable</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">eval_budget</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">None</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">error_metrics</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="n"><spanclass="pre">Iterable</span><spanclass="p"><spanclass="pre">[</span></span><spanclass="pre">Union</span><spanclass="p"><spanclass="pre">[</span></span><spanclass="pre">str</span><spanclass="p"><spanclass="pre">,</span></span><spanclass="pre">Callable</span><spanclass="p"><spanclass="pre">]</span></span><spanclass="p"><spanclass="pre">]</span></span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">'mae'</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.evaluation.gen_prevalence_report"title="Permalink to this definition">¶</a></dt>
<dd><p>GGenerates an evaluation report for a custom protocol defined as a generator function that yields
samples at each iteration. The sequence of samples is processed exhaustively if <cite>eval_budget=None</cite>
or up to the <cite>eval_budget</cite> iterations if specified.
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.evaluation.</span></span><spanclass="sig-name descname"><spanclass="pre">natural_prevalence_protocol</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">model</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="n"><aclass="reference internal"href="quapy.method.html#quapy.method.base.BaseQuantifier"title="quapy.method.base.BaseQuantifier"><spanclass="pre">quapy.method.base.BaseQuantifier</span></a></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">test</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="n"><aclass="reference internal"href="quapy.data.html#quapy.data.base.LabelledCollection"title="quapy.data.base.LabelledCollection"><spanclass="pre">quapy.data.base.LabelledCollection</span></a></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">sample_size</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">repeats</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">1</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">n_jobs</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">1</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">random_seed</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">42</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">error_metric</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="n"><spanclass="pre">Union</span><spanclass="p"><spanclass="pre">[</span></span><spanclass="pre">str</span><spanclass="p"><spanclass="pre">,</span></span><spanclass="pre">Callable</span><spanclass="p"><spanclass="pre">]</span></span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">'mae'</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">verbose</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">False</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.evaluation.natural_prevalence_protocol"title="Permalink to this definition">¶</a></dt>
<dd><p>Generates samples according to the Natural Prevalence Protocol (NPP).
The NPP consists of drawing samples uniformly at random, therefore approximately preserving the natural
prevalence of the collection.</p>
<dlclass="field-list simple">
<dtclass="field-odd">Parameters</dt>
<ddclass="field-odd"><ulclass="simple">
<li><p><strong>model</strong>– the model in charge of generating the class prevalence estimations</p></li>
<li><p><strong>test</strong>– the test set on which to perform NPP</p></li>
<li><p><strong>sample_size</strong>– integer, the size of the samples</p></li>
<li><p><strong>repeats</strong>– integer, the number of samples to generate</p></li>
<li><p><strong>n_jobs</strong>– integer, number of jobs to be run in parallel (default 1)</p></li>
<li><p><strong>random_seed</strong>– allows to replicate the samplings. The seed is local to the method and does not affect
any other random process (default 42)</p></li>
<li><p><strong>error_metric</strong>– a string indicating the name of the error (as defined in <aclass="reference internal"href="#module-quapy.error"title="quapy.error"><codeclass="xref py py-mod docutils literal notranslate"><spanclass="pre">quapy.error</span></code></a>) or a
callable error function</p></li>
<li><p><strong>verbose</strong>– if True, shows a progress bar</p></li>
</ul>
</dd>
<dtclass="field-even">Returns</dt>
<ddclass="field-even"><p>yields one sample at a time</p>
in which the rows correspond to different samples, and the columns inform of the true prevalence values,
the estimated prevalence values, and the score obtained by each of the evaluation measures indicated.</p>
<dlclass="field-list simple">
<dtclass="field-odd">Parameters</dt>
<ddclass="field-odd"><ulclass="simple">
<li><p><strong>model</strong>– the model in charge of generating the class prevalence estimations</p></li>
<li><p><strong>test</strong>– the test set on which to perform NPP</p></li>
<li><p><strong>sample_size</strong>– integer, the size of the samples</p></li>
<li><p><strong>repeats</strong>– integer, the number of samples to generate</p></li>
<li><p><strong>n_jobs</strong>– integer, number of jobs to be run in parallel (default 1)</p></li>
<li><p><strong>random_seed</strong>– allows to replicate the samplings. The seed is local to the method and does not affect
any other random process (default 42)</p></li>
<li><p><strong>error_metrics</strong>– a string indicating the name of the error (as defined in <aclass="reference internal"href="#module-quapy.error"title="quapy.error"><codeclass="xref py py-mod docutils literal notranslate"><spanclass="pre">quapy.error</span></code></a>) or a
callable error function; optionally, a list of strings or callables can be indicated, if the results
are to be evaluated with more than one error metric. Default is “mae”</p></li>
<li><p><strong>verbose</strong>– if True, shows a progress bar</p></li>
</ul>
</dd>
<dtclass="field-even">Returns</dt>
<ddclass="field-even"><p>a tuple containing two <cite>np.ndarrays</cite> of shape <cite>(m,n,)</cite> with <cite>m</cite> the number of samples
<cite>(repeats)</cite> and <cite>n</cite> the number of classes. The first one contains the true prevalence values
for the samples generated while the second one contains the prevalence estimations</p>
<spanid="quapy-functional-module"></span><h2>quapy.functional module<aclass="headerlink"href="#module-quapy.functional"title="Permalink to this headline">¶</a></h2>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.functional.</span></span><spanclass="sig-name descname"><spanclass="pre">HellingerDistance</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">P</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">Q</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.functional.HellingerDistance"title="Permalink to this definition">¶</a></dt>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.functional.</span></span><spanclass="sig-name descname"><spanclass="pre">adjusted_quantification</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">prevalence_estim</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">tpr</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">fpr</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">clip</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">True</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.functional.adjusted_quantification"title="Permalink to this definition">¶</a></dt>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.functional.</span></span><spanclass="sig-name descname"><spanclass="pre">artificial_prevalence_sampling</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">dimensions</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">n_prevalences</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">21</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">repeat</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">1</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">return_constrained_dim</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">False</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.functional.artificial_prevalence_sampling"title="Permalink to this definition">¶</a></dt>
<ddclass="field-even"><p>a <cite>np.ndarray</cite> of shape <cite>(n, dimensions)</cite> if <cite>return_constrained_dim=True</cite> or of shape <cite>(n, dimensions-1)</cite>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.functional.</span></span><spanclass="sig-name descname"><spanclass="pre">get_nprevpoints_approximation</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">combinations_budget</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="n"><spanclass="pre">int</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">n_classes</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="n"><spanclass="pre">int</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">n_repeats</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="n"><spanclass="pre">int</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">1</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.functional.get_nprevpoints_approximation"title="Permalink to this definition">¶</a></dt>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.functional.</span></span><spanclass="sig-name descname"><spanclass="pre">normalize_prevalence</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">prevalences</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.functional.normalize_prevalence"title="Permalink to this definition">¶</a></dt>
<dd><p>Normalize a vector or matrix of prevalence values. The normalization consists of applying a L1 normalization in
cases in which the prevalence values are not all-zeros, and to convert the prevalence values into <cite>1/n_classes</cite> in
cases in which all values are zero.</p>
<dlclass="field-list simple">
<dtclass="field-odd">Parameters</dt>
<ddclass="field-odd"><p><strong>prevalences</strong>– array-like of shape <cite>(n_classes,)</cite> or of shape <cite>(n_samples, n_classes,)</cite> with prevalence values</p>
</dd>
<dtclass="field-even">Returns</dt>
<ddclass="field-even"><p>a normalized vector or matrix of prevalence values</p>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.functional.</span></span><spanclass="sig-name descname"><spanclass="pre">num_prevalence_combinations</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">n_prevpoints</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="n"><spanclass="pre">int</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">n_classes</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="n"><spanclass="pre">int</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">n_repeats</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="n"><spanclass="pre">int</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">1</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.functional.num_prevalence_combinations"title="Permalink to this definition">¶</a></dt>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.functional.</span></span><spanclass="sig-name descname"><spanclass="pre">prevalence_from_labels</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">labels</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">classes</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.functional.prevalence_from_labels"title="Permalink to this definition">¶</a></dt>
<dd><p>Computed the prevalence values from a vector of labels.</p>
<dlclass="field-list simple">
<dtclass="field-odd">Parameters</dt>
<ddclass="field-odd"><ulclass="simple">
<li><p><strong>labels</strong>– array-like of shape <cite>(n_instances)</cite> with the label for each instance</p></li>
<li><p><strong>classes</strong>– the class labels. This is needed in order to correctly compute the prevalence vector even when
some classes have no examples.</p></li>
</ul>
</dd>
<dtclass="field-even">Returns</dt>
<ddclass="field-even"><p>an ndarray of shape <cite>(len(classes))</cite> with the class prevalence values</p>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.functional.</span></span><spanclass="sig-name descname"><spanclass="pre">prevalence_from_probabilities</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">posteriors</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">binarize</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="n"><spanclass="pre">bool</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">False</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.functional.prevalence_from_probabilities"title="Permalink to this definition">¶</a></dt>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.functional.</span></span><spanclass="sig-name descname"><spanclass="pre">prevalence_linspace</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">n_prevalences</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">21</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">repeats</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">1</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">smooth_limits_epsilon</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">0.01</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.functional.prevalence_linspace"title="Permalink to this definition">¶</a></dt>
<dd><p>Produces an array of uniformly separated values of prevalence.
By default, produces an array of 21 prevalence values, with
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.functional.</span></span><spanclass="sig-name descname"><spanclass="pre">strprev</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">prevalences</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">prec</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">3</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.functional.strprev"title="Permalink to this definition">¶</a></dt>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.functional.</span></span><spanclass="sig-name descname"><spanclass="pre">uniform_prevalence_sampling</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">n_classes</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">size</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">1</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.functional.uniform_prevalence_sampling"title="Permalink to this definition">¶</a></dt>
<li><p><strong>n_classes</strong>– integer, number of classes (dimensionality of the simplex)</p></li>
<li><p><strong>size</strong>– number of samples to return</p></li>
</ul>
</dd>
<dtclass="field-even">Returns</dt>
<ddclass="field-even"><p><cite>np.ndarray</cite> of shape <cite>(size, n_classes,)</cite> if <cite>size>1</cite>, or of shape <cite>(n_classes,)</cite> otherwise</p>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.functional.</span></span><spanclass="sig-name descname"><spanclass="pre">uniform_simplex_sampling</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">n_classes</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">size</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">1</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.functional.uniform_simplex_sampling"title="Permalink to this definition">¶</a></dt>
<li><p><strong>n_classes</strong>– integer, number of classes (dimensionality of the simplex)</p></li>
<li><p><strong>size</strong>– number of samples to return</p></li>
</ul>
</dd>
<dtclass="field-even">Returns</dt>
<ddclass="field-even"><p><cite>np.ndarray</cite> of shape <cite>(size, n_classes,)</cite> if <cite>size>1</cite>, or of shape <cite>(n_classes,)</cite> otherwise</p>
<spanid="quapy-model-selection-module"></span><h2>quapy.model_selection module<aclass="headerlink"href="#module-quapy.model_selection"title="Permalink to this headline">¶</a></h2>
<emclass="property"><spanclass="pre">class</span></em><spanclass="sig-prename descclassname"><spanclass="pre">quapy.model_selection.</span></span><spanclass="sig-name descname"><spanclass="pre">GridSearchQ</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="pre">model:</span><spanclass="pre">quapy.method.base.BaseQuantifier,</span><spanclass="pre">param_grid:</span><spanclass="pre">dict,</span><spanclass="pre">sample_size:</span><spanclass="pre">Optional[int],</span><spanclass="pre">protocol='app',</span><spanclass="pre">n_prevpoints:</span><spanclass="pre">Optional[int]</span><spanclass="pre">=</span><spanclass="pre">None,</span><spanclass="pre">n_repetitions:</span><spanclass="pre">int</span><spanclass="pre">=</span><spanclass="pre">1,</span><spanclass="pre">eval_budget:</span><spanclass="pre">Optional[int]</span><spanclass="pre">=</span><spanclass="pre">None,</span><spanclass="pre">error:</span><spanclass="pre">Union[Callable,</span><spanclass="pre">str]</span><spanclass="pre">=</span><spanclass="pre"><function</span><spanclass="pre">mae>,</span><spanclass="pre">refit=True,</span><spanclass="pre">val_split=0.4,</span><spanclass="pre">n_jobs=1,</span><spanclass="pre">random_seed=42,</span><spanclass="pre">timeout=-1,</span><spanclass="pre">verbose=False</span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.model_selection.GridSearchQ"title="Permalink to this definition">¶</a></dt>
<p>Grid Search optimization targeting a quantification-oriented metric.</p>
<p>Optimizes the hyperparameters of a quantification method, based on an evaluation method and on an evaluation
protocol for quantification.</p>
<dlclass="field-list simple">
<dtclass="field-odd">Parameters</dt>
<ddclass="field-odd"><ulclass="simple">
<li><p><strong>model</strong> (<aclass="reference internal"href="quapy.method.html#quapy.method.base.BaseQuantifier"title="quapy.method.base.BaseQuantifier"><em>BaseQuantifier</em></a>) – the quantifier to optimize</p></li>
<li><p><strong>param_grid</strong>– a dictionary with keys the parameter names and values the list of values to explore</p></li>
<li><p><strong>sample_size</strong>– the size of the samples to extract from the validation set (ignored if protocl=’gen’)</p></li>
<li><p><strong>protocol</strong>– either ‘app’ for the artificial prevalence protocol, ‘npp’ for the natural prevalence
protocol, or ‘gen’ for using a custom sampling generator function</p></li>
<li><p><strong>n_prevpoints</strong>– if specified, indicates the number of equally distant points to extract from the interval
[0,1] in order to define the prevalences of the samples; e.g., if n_prevpoints=5, then the prevalences for
each class will be explored in [0.00, 0.25, 0.50, 0.75, 1.00]. If not specified, then eval_budget is requested.
Ignored if protocol!=’app’.</p></li>
<li><p><strong>n_repetitions</strong>– the number of repetitions for each combination of prevalences. This parameter is ignored
for the protocol=’app’ if eval_budget is set and is lower than the number of combinations that would be
generated using the value assigned to n_prevpoints (for the current number of classes and n_repetitions).
Ignored for protocol=’npp’ and protocol=’gen’ (use eval_budget for setting a maximum number of samples in
those cases).</p></li>
<li><p><strong>eval_budget</strong>– if specified, sets a ceil on the number of evaluations to perform for each hyper-parameter
combination. For example, if protocol=’app’, there are 3 classes, n_repetitions=1 and eval_budget=20, then
n_prevpoints will be set to 5, since this will generate 15 different prevalences, i.e., [0, 0, 1],
[0, 0.25, 0.75], [0, 0.5, 0.5] … [1, 0, 0], and since setting it to 6 would generate more than
20. When protocol=’gen’, indicates the maximum number of samples to generate, but less samples will be
generated if the generator yields less samples.</p></li>
<li><p><strong>error</strong>– an error function (callable) or a string indicating the name of an error function (valid ones
are those in qp.error.QUANTIFICATION_ERROR</p></li>
<li><p><strong>refit</strong>– whether or not to refit the model on the whole labelled collection (training+validation) with
the best chosen hyperparameter combination. Ignored if protocol=’gen’</p></li>
<li><p><strong>val_split</strong>– either a LabelledCollection on which to test the performance of the different settings, or
a float in [0,1] indicating the proportion of labelled data to extract from the training set, or a callable
returning a generator function each time it is invoked (only for protocol=’gen’).</p></li>
<li><p><strong>n_jobs</strong>– number of parallel jobs</p></li>
<li><p><strong>random_seed</strong>– set the seed of the random generator to replicate experiments. Ignored if protocol=’gen’.</p></li>
<li><p><strong>timeout</strong>– establishes a timer (in seconds) for each of the hyperparameters configurations being tested.
Whenever a run takes longer than this timer, that configuration will be ignored. If all configurations end up
being ignored, a TimeoutError exception is raised. If -1 (default) then no time bound is set.</p></li>
<li><p><strong>verbose</strong>– set to True to get information through the stdout</p></li>
<spanclass="sig-name descname"><spanclass="pre">best_model</span></span><spanclass="sig-paren">(</span><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.model_selection.GridSearchQ.best_model"title="Permalink to this definition">¶</a></dt>
<dd><p>Returns the best model found after calling the <aclass="reference internal"href="#quapy.model_selection.GridSearchQ.fit"title="quapy.model_selection.GridSearchQ.fit"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">fit()</span></code></a> method, i.e., the one trained on the combination
of hyper-parameters that minimized the error function.</p>
<emclass="property"><spanclass="pre">property</span></em><spanclass="sig-name descname"><spanclass="pre">classes_</span></span><aclass="headerlink"href="#quapy.model_selection.GridSearchQ.classes_"title="Permalink to this definition">¶</a></dt>
<spanclass="sig-name descname"><spanclass="pre">fit</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">training</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="n"><aclass="reference internal"href="quapy.data.html#quapy.data.base.LabelledCollection"title="quapy.data.base.LabelledCollection"><spanclass="pre">quapy.data.base.LabelledCollection</span></a></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">val_split</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="n"><spanclass="pre">Optional</span><spanclass="p"><spanclass="pre">[</span></span><spanclass="pre">Union</span><spanclass="p"><spanclass="pre">[</span></span><aclass="reference internal"href="quapy.data.html#quapy.data.base.LabelledCollection"title="quapy.data.base.LabelledCollection"><spanclass="pre">quapy.data.base.LabelledCollection</span></a><spanclass="p"><spanclass="pre">,</span></span><spanclass="pre">float</span><spanclass="p"><spanclass="pre">,</span></span><spanclass="pre">Callable</span><spanclass="p"><spanclass="pre">]</span></span><spanclass="p"><spanclass="pre">]</span></span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">None</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.model_selection.GridSearchQ.fit"title="Permalink to this definition">¶</a></dt>
<dd><dlclass="simple">
<dt>Learning routine. Fits methods with all combinations of hyperparameters and selects the one minimizing</dt><dd><p>the error metric.</p>
</dd>
</dl>
<dlclass="field-list simple">
<dtclass="field-odd">Parameters</dt>
<ddclass="field-odd"><ulclass="simple">
<li><p><strong>training</strong>– the training set on which to optimize the hyperparameters</p></li>
<li><p><strong>val_split</strong>– either a LabelledCollection on which to test the performance of the different settings, or
a float in [0,1] indicating the proportion of labelled data to extract from the training set</p></li>
<spanclass="sig-name descname"><spanclass="pre">get_params</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">deep</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">True</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.model_selection.GridSearchQ.get_params"title="Permalink to this definition">¶</a></dt>
<dd><p>Returns the dictionary of hyper-parameters to explore (<cite>param_grid</cite>)</p>
<spanclass="sig-name descname"><spanclass="pre">quantify</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">instances</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.model_selection.GridSearchQ.quantify"title="Permalink to this definition">¶</a></dt>
<dd><p>Estimate class prevalence values using the best model found after calling the <aclass="reference internal"href="#quapy.model_selection.GridSearchQ.fit"title="quapy.model_selection.GridSearchQ.fit"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">fit()</span></code></a> method.</p>
<spanclass="sig-name descname"><spanclass="pre">set_params</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="o"><spanclass="pre">**</span></span><spanclass="n"><spanclass="pre">parameters</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.model_selection.GridSearchQ.set_params"title="Permalink to this definition">¶</a></dt>
<dd><p>Sets the hyper-parameters to explore.</p>
<dlclass="field-list simple">
<dtclass="field-odd">Parameters</dt>
<ddclass="field-odd"><p><strong>parameters</strong>– a dictionary with keys the parameter names and values the list of values to explore</p>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.plot.</span></span><spanclass="sig-name descname"><spanclass="pre">binary_diagonal</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">method_names</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">true_prevs</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">estim_prevs</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">pos_class</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">1</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">title</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">None</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">show_std</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">True</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">legend</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">True</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">train_prev</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">None</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">savepath</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">None</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">method_order</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">None</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.plot.binary_diagonal"title="Permalink to this definition">¶</a></dt>
<dd><p>The diagonal plot displays the predicted prevalence values (along the y-axis) as a function of the true prevalence
values (along the x-axis). The optimal quantifier is described by the diagonal (0,0)-(1,1) of the plot (hence the
name). It is convenient for binary quantification problems, though it can be used for multiclass problems by
indicating which class is to be taken as the positive class. (For multiclass quantification problems, other plots
like the <aclass="reference internal"href="#quapy.plot.error_by_drift"title="quapy.plot.error_by_drift"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">error_by_drift()</span></code></a> might be preferable though).</p>
<dlclass="field-list simple">
<dtclass="field-odd">Parameters</dt>
<ddclass="field-odd"><ulclass="simple">
<li><p><strong>method_names</strong>– array-like with the method names for each experiment</p></li>
<li><p><strong>true_prevs</strong>– array-like with the true prevalence values (each being a ndarray with n_classes components) for
each experiment</p></li>
<li><p><strong>estim_prevs</strong>– array-like with the estimated prevalence values (each being a ndarray with n_classes components)
for each experiment</p></li>
<li><p><strong>pos_class</strong>– index of the positive class</p></li>
<li><p><strong>title</strong>– the title to be displayed in the plot</p></li>
<li><p><strong>show_std</strong>– whether or not to show standard deviations (represented by color bands). This might be inconvenient
for cases in which many methods are compared, or when the standard deviations are high – default True)</p></li>
<li><p><strong>legend</strong>– whether or not to display the leyend (default True)</p></li>
<li><p><strong>train_prev</strong>– if indicated (default is None), the training prevalence (for the positive class) is hightlighted
in the plot. This is convenient when all the experiments have been conducted in the same dataset.</p></li>
<li><p><strong>savepath</strong>– path where to save the plot. If not indicated (as default), the plot is shown.</p></li>
<li><p><strong>method_order</strong>– if indicated (default is None), imposes the order in which the methods are processed (i.e.,
listed in the legend and associated with matplotlib colors).</p></li>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.plot.</span></span><spanclass="sig-name descname"><spanclass="pre">error_by_drift</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">method_names</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">true_prevs</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">estim_prevs</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">tr_prevs</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">n_bins</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">20</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">error_name</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">'ae'</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">show_std</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">False</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">show_density</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">True</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">logscale</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">False</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">title</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">'Quantification</span><spanclass="pre">error</span><spanclass="pre">as</span><spanclass="pre">a</span><spanclass="pre">function</span><spanclass="pre">of</span><spanclass="pre">distribution</span><spanclass="pre">shift'</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">vlines</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">None</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">method_order</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">None</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">savepath</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">None</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.plot.error_by_drift"title="Permalink to this definition">¶</a></dt>
<dd><p>Plots the error (along the x-axis, as measured in terms of <cite>error_name</cite>) as a function of the train-test shift
(along the y-axis, as measured in terms of <aclass="reference internal"href="#quapy.error.ae"title="quapy.error.ae"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">quapy.error.ae()</span></code></a>). This plot is useful especially for multiclass
problems, in which “diagonal plots” may be cumbersone, and in order to gain understanding about how methods
fare in different regions of the prior probability shift spectrum (e.g., in the low-shift regime vs. in the
high-shift regime).</p>
<dlclass="field-list simple">
<dtclass="field-odd">Parameters</dt>
<ddclass="field-odd"><ulclass="simple">
<li><p><strong>method_names</strong>– array-like with the method names for each experiment</p></li>
<li><p><strong>true_prevs</strong>– array-like with the true prevalence values (each being a ndarray with n_classes components) for
each experiment</p></li>
<li><p><strong>estim_prevs</strong>– array-like with the estimated prevalence values (each being a ndarray with n_classes components)
for each experiment</p></li>
<li><p><strong>tr_prevs</strong>– training prevalence of each experiment</p></li>
<li><p><strong>n_bins</strong>– number of bins in which the y-axis is to be divided (default is 20)</p></li>
<li><p><strong>error_name</strong>– a string representing the name of an error function (as defined in <cite>quapy.error</cite>, default is “ae”)</p></li>
<li><p><strong>show_std</strong>– whether or not to show standard deviations as color bands (default is False)</p></li>
<li><p><strong>show_density</strong>– whether or not to display the distribution of experiments for each bin (default is True)</p></li>
<li><p><strong>logscale</strong>– whether or not to log-scale the y-error measure (default is False)</p></li>
<li><p><strong>title</strong>– title of the plot (default is “Quantification error as a function of distribution shift”)</p></li>
<li><p><strong>vlines</strong>– array-like list of values (default is None). If indicated, highlights some regions of the space
using vertical dotted lines.</p></li>
<li><p><strong>method_order</strong>– if indicated (default is None), imposes the order in which the methods are processed (i.e.,
listed in the legend and associated with matplotlib colors).</p></li>
<li><p><strong>savepath</strong>– path where to save the plot. If not indicated (as default), the plot is shown.</p></li>
<emclass="property"><spanclass="pre">class</span></em><spanclass="sig-prename descclassname"><spanclass="pre">quapy.util.</span></span><spanclass="sig-name descname"><spanclass="pre">EarlyStop</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">patience</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">lower_is_better</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">True</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.util.EarlyStop"title="Permalink to this definition">¶</a></dt>
<spanclass="gp">>>></span><spanclass="n">earlystop</span><spanclass="o">.</span><spanclass="n">IMPROVED</span><spanclass="c1"># is True</span>
<spanclass="gp">>>></span><spanclass="n">earlystop</span><spanclass="o">.</span><spanclass="n">STOP</span><spanclass="c1"># is False (patience=1)</span>
<spanclass="gp">>>></span><spanclass="n">earlystop</span><spanclass="o">.</span><spanclass="n">STOP</span><spanclass="c1"># is True (patience=0)</span>
<spanclass="gp">>>></span><spanclass="n">earlystop</span><spanclass="o">.</span><spanclass="n">best_epoch</span><spanclass="c1"># is 1</span>
<spanclass="gp">>>></span><spanclass="n">earlystop</span><spanclass="o">.</span><spanclass="n">best_score</span><spanclass="c1"># is 0.7</span>
</pre></div>
</div>
<dlclass="field-list simple">
<dtclass="field-odd">Parameters</dt>
<ddclass="field-odd"><p><strong>lower_is_better</strong>– if True (default) the metric is to be minimized.</p>
</dd>
<dtclass="field-even">Variables</dt>
<ddclass="field-even"><ulclass="simple">
<li><p><strong>best_score</strong>– keeps track of the best value seen so far</p></li>
<li><p><strong>best_epoch</strong>– keeps track of the epoch in which the best score was set</p></li>
<li><p><strong>STOP</strong>– flag (boolean) indicating the stopping condition</p></li>
<li><p><strong>IMPROVED</strong>– flag (boolean) indicating whether there was an improvement in the last call</p></li>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.util.</span></span><spanclass="sig-name descname"><spanclass="pre">create_if_not_exist</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">path</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.util.create_if_not_exist"title="Permalink to this definition">¶</a></dt>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.util.</span></span><spanclass="sig-name descname"><spanclass="pre">create_parent_dir</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">path</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.util.create_parent_dir"title="Permalink to this definition">¶</a></dt>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.util.</span></span><spanclass="sig-name descname"><spanclass="pre">download_file</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">url</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">archive_filename</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.util.download_file"title="Permalink to this definition">¶</a></dt>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.util.</span></span><spanclass="sig-name descname"><spanclass="pre">download_file_if_not_exists</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">url</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">archive_filename</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.util.download_file_if_not_exists"title="Permalink to this definition">¶</a></dt>
<dd><p>Dowloads a function (using <aclass="reference internal"href="#quapy.util.download_file"title="quapy.util.download_file"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">download_file()</span></code></a>) if the file does not exist.</p>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.util.</span></span><spanclass="sig-name descname"><spanclass="pre">get_quapy_home</span></span><spanclass="sig-paren">(</span><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.util.get_quapy_home"title="Permalink to this definition">¶</a></dt>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.util.</span></span><spanclass="sig-name descname"><spanclass="pre">map_parallel</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">func</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">args</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">n_jobs</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.util.map_parallel"title="Permalink to this definition">¶</a></dt>
<dd><p>Applies func to n_jobs slices of args. E.g., if args is an array of 99 items and n_jobs=2, then
func is applied in two parallel processes to args[0:50] and to args[50:99]</p>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.util.</span></span><spanclass="sig-name descname"><spanclass="pre">parallel</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">func</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">args</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">n_jobs</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.util.parallel"title="Permalink to this definition">¶</a></dt>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.util.</span></span><spanclass="sig-name descname"><spanclass="pre">pickled_resource</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">pickle_path</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="n"><spanclass="pre">str</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">generation_func</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="n"><spanclass="pre">callable</span></span></em>, <emclass="sig-param"><spanclass="o"><spanclass="pre">*</span></span><spanclass="n"><spanclass="pre">args</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.util.pickled_resource"title="Permalink to this definition">¶</a></dt>
<dd><p>Allows for fast reuse of resources that are generated only once by calling generation_func(<ahref="#id4"><spanclass="problematic"id="id5">*</span></a>args). The next times
this function is invoked, it loads the pickled resource. Example:</p>
<divclass="doctest highlight-default notranslate"><divclass="highlight"><pre><span></span><spanclass="gp">>>></span><spanclass="k">def</span><spanclass="nf">some_array</span><spanclass="p">(</span><spanclass="n">n</span><spanclass="p">):</span><spanclass="c1"># a mock resource created with one parameter (`n`)</span>
<spanclass="gp">>>></span><spanclass="n">pickled_resource</span><spanclass="p">(</span><spanclass="s1">'./my_array.pkl'</span><spanclass="p">,</span><spanclass="n">some_array</span><spanclass="p">,</span><spanclass="mi">10</span><spanclass="p">)</span><spanclass="c1"># the resource does not exist: it is created by calling some_array(10)</span>
<spanclass="gp">>>></span><spanclass="n">pickled_resource</span><spanclass="p">(</span><spanclass="s1">'./my_array.pkl'</span><spanclass="p">,</span><spanclass="n">some_array</span><spanclass="p">,</span><spanclass="mi">10</span><spanclass="p">)</span><spanclass="c1"># the resource exists; it is loaded from './my_array.pkl'</span>
</pre></div>
</div>
<dlclass="field-list simple">
<dtclass="field-odd">Parameters</dt>
<ddclass="field-odd"><ulclass="simple">
<li><p><strong>pickle_path</strong>– the path where to save (first time) and load (next times) the resource</p></li>
<li><p><strong>generation_func</strong>– the function that generates the resource, in case it does not exist in pickle_path</p></li>
<li><p><strong>args</strong>– any arg that generation_func uses for generating the resources</p></li>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.util.</span></span><spanclass="sig-name descname"><spanclass="pre">save_text_file</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">path</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">text</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.util.save_text_file"title="Permalink to this definition">¶</a></dt>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.util.</span></span><spanclass="sig-name descname"><spanclass="pre">temp_seed</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">seed</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.util.temp_seed"title="Permalink to this definition">¶</a></dt>
<spanid="module-contents"></span><h2>Module contents<aclass="headerlink"href="#module-quapy"title="Permalink to this headline">¶</a></h2>
<dlclass="py function">
<dtclass="sig sig-object py"id="quapy.isbinary">
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.</span></span><spanclass="sig-name descname"><spanclass="pre">isbinary</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">x</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.isbinary"title="Permalink to this definition">¶</a></dt>
