<spanclass="sig-prename descclassname"><spanclass="pre">quapy.error.</span></span><spanclass="sig-name descname"><spanclass="pre">absolute_error</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">prevs</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">prevs_hat</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.error.absolute_error"title="Permalink to this definition">¶</a></dt>
<dd><dlclass="simple">
<dt>Computes the absolute error between the two prevalence vectors.</dt><dd><p>Absolute error between two prevalence vectors <spanclass="math notranslate nohighlight">\(p\)</span> and <spanclass="math notranslate nohighlight">\(\hat{p}\)</span> is computed as
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.error.</span></span><spanclass="sig-name descname"><spanclass="pre">acc_error</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">y_true</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">y_pred</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.error.acc_error"title="Permalink to this definition">¶</a></dt>
<dd><p>Computes the error in terms of 1-accuracy. The accuracy is computed as <spanclass="math notranslate nohighlight">\(\frac{tp+tn}{tp+fp+fn+tn}\)</span>, with
<cite>tp</cite>, <cite>fp</cite>, <cite>fn</cite>, and <cite>tn</cite> standing for true positives, false positives, false negatives, and true negatives,
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.error.</span></span><spanclass="sig-name descname"><spanclass="pre">acce</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">y_true</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">y_pred</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.error.acce"title="Permalink to this definition">¶</a></dt>
<dd><p>Computes the error in terms of 1-accuracy. The accuracy is computed as <spanclass="math notranslate nohighlight">\(\frac{tp+tn}{tp+fp+fn+tn}\)</span>, with
<cite>tp</cite>, <cite>fp</cite>, <cite>fn</cite>, and <cite>tn</cite> standing for true positives, false positives, false negatives, and true negatives,
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.error.</span></span><spanclass="sig-name descname"><spanclass="pre">ae</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">prevs</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">prevs_hat</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.error.ae"title="Permalink to this definition">¶</a></dt>
<dd><dlclass="simple">
<dt>Computes the absolute error between the two prevalence vectors.</dt><dd><p>Absolute error between two prevalence vectors <spanclass="math notranslate nohighlight">\(p\)</span> and <spanclass="math notranslate nohighlight">\(\hat{p}\)</span> is computed as
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.error.</span></span><spanclass="sig-name descname"><spanclass="pre">f1_error</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">y_true</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">y_pred</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.error.f1_error"title="Permalink to this definition">¶</a></dt>
<dd><p>F1 error: simply computes the error in terms of macro <spanclass="math notranslate nohighlight">\(F_1\)</span>, i.e., <spanclass="math notranslate nohighlight">\(1-F_1^M\)</span>,
where <spanclass="math notranslate nohighlight">\(F_1\)</span> is the harmonic mean of precision and recall, defined as <spanclass="math notranslate nohighlight">\(\frac{2tp}{2tp+fp+fn}\)</span>,
with <cite>tp</cite>, <cite>fp</cite>, and <cite>fn</cite> standing for true positives, false positives, and false negatives, respectively.
<cite>Macro</cite> averaging means the <spanclass="math notranslate nohighlight">\(F_1\)</span> is computed for each category independently, and then averaged.</p>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.error.</span></span><spanclass="sig-name descname"><spanclass="pre">f1e</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">y_true</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">y_pred</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.error.f1e"title="Permalink to this definition">¶</a></dt>
<dd><p>F1 error: simply computes the error in terms of macro <spanclass="math notranslate nohighlight">\(F_1\)</span>, i.e., <spanclass="math notranslate nohighlight">\(1-F_1^M\)</span>,
where <spanclass="math notranslate nohighlight">\(F_1\)</span> is the harmonic mean of precision and recall, defined as <spanclass="math notranslate nohighlight">\(\frac{2tp}{2tp+fp+fn}\)</span>,
with <cite>tp</cite>, <cite>fp</cite>, and <cite>fn</cite> standing for true positives, false positives, and false negatives, respectively.
<cite>Macro</cite> averaging means the <spanclass="math notranslate nohighlight">\(F_1\)</span> is computed for each category independently, and then averaged.</p>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.error.</span></span><spanclass="sig-name descname"><spanclass="pre">from_name</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">err_name</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.error.from_name"title="Permalink to this definition">¶</a></dt>
<dd><p>Gets an error function from its name. E.g., <cite>from_name(“mae”)</cite> will return function <aclass="reference internal"href="#quapy.error.mae"title="quapy.error.mae"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">quapy.error.mae()</span></code></a></p>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.error.</span></span><spanclass="sig-name descname"><spanclass="pre">kld</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">p</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">p_hat</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">eps</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">None</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.error.kld"title="Permalink to this definition">¶</a></dt>
<dt>Computes the Kullback-Leibler divergence between the two prevalence distributions.</dt><dd><p>Kullback-Leibler divergence between two prevalence distributions <spanclass="math notranslate nohighlight">\(p\)</span> and <spanclass="math notranslate nohighlight">\(\hat{p}\)</span> is computed as
<spanclass="math notranslate nohighlight">\(KLD(p,\hat{p})=D_{KL}(p||\hat{p})=\sum_{y\in \mathcal{Y}} p(y)\log\frac{p(y)}{\hat{p}(y)}\)</span>, where
<spanclass="math notranslate nohighlight">\(\mathcal{Y}\)</span> are the classes of interest.
The distributions are smoothed using the <cite>eps</cite> factor (see <aclass="reference internal"href="#quapy.error.smooth"title="quapy.error.smooth"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">quapy.error.smooth()</span></code></a>).</p>
<li><p><strong>prevs</strong>– array-like of shape <cite>(n_classes,)</cite> with the true prevalence values</p></li>
<li><p><strong>prevs_hat</strong>– array-like of shape <cite>(n_classes,)</cite> with the predicted prevalence values</p></li>
<li><p><strong>eps</strong>– smoothing factor. KLD is not defined in cases in which the distributions contain zeros; <cite>eps</cite>
is typically set to be <spanclass="math notranslate nohighlight">\(\frac{1}{2T}\)</span>, with <spanclass="math notranslate nohighlight">\(T\)</span> the sample size. If <cite>eps=None</cite>, the sample size
will be taken from the environment variable <cite>SAMPLE_SIZE</cite> (which has thus to be set beforehand).</p></li>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.error.</span></span><spanclass="sig-name descname"><spanclass="pre">mae</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">prevs</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">prevs_hat</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.error.mae"title="Permalink to this definition">¶</a></dt>
<dd><p>Computes the mean absolute error (see <aclass="reference internal"href="#quapy.error.ae"title="quapy.error.ae"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">quapy.error.ae()</span></code></a>) across the sample pairs.</p>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.error.</span></span><spanclass="sig-name descname"><spanclass="pre">mean_absolute_error</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">prevs</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">prevs_hat</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.error.mean_absolute_error"title="Permalink to this definition">¶</a></dt>
<dd><p>Computes the mean absolute error (see <aclass="reference internal"href="#quapy.error.ae"title="quapy.error.ae"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">quapy.error.ae()</span></code></a>) across the sample pairs.</p>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.error.</span></span><spanclass="sig-name descname"><spanclass="pre">mean_relative_absolute_error</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">p</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">p_hat</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">eps</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">None</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.error.mean_relative_absolute_error"title="Permalink to this definition">¶</a></dt>
<dd><p>Computes the mean relative absolute error (see <aclass="reference internal"href="#quapy.error.rae"title="quapy.error.rae"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">quapy.error.rae()</span></code></a>) across the sample pairs.
The distributions are smoothed using the <cite>eps</cite> factor (see <aclass="reference internal"href="#quapy.error.smooth"title="quapy.error.smooth"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">quapy.error.smooth()</span></code></a>).</p>
<li><p><strong>prevs</strong>– array-like of shape <cite>(n_samples, n_classes,)</cite> with the true prevalence values</p></li>
<li><p><strong>prevs_hat</strong>– array-like of shape <cite>(n_samples, n_classes,)</cite> with the predicted prevalence values</p></li>
<li><p><strong>eps</strong>– smoothing factor. <cite>mrae</cite> is not defined in cases in which the true distribution contains zeros; <cite>eps</cite>
is typically set to be <spanclass="math notranslate nohighlight">\(\frac{1}{2T}\)</span>, with <spanclass="math notranslate nohighlight">\(T\)</span> the sample size. If <cite>eps=None</cite>, the sample size
will be taken from the environment variable <cite>SAMPLE_SIZE</cite> (which has thus to be set beforehand).</p></li>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.error.</span></span><spanclass="sig-name descname"><spanclass="pre">mkld</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">prevs</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">prevs_hat</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">eps</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">None</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.error.mkld"title="Permalink to this definition">¶</a></dt>
<dd><p>Computes the mean Kullback-Leibler divergence (see <aclass="reference internal"href="#quapy.error.kld"title="quapy.error.kld"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">quapy.error.kld()</span></code></a>) across the sample pairs.
The distributions are smoothed using the <cite>eps</cite> factor (see <aclass="reference internal"href="#quapy.error.smooth"title="quapy.error.smooth"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">quapy.error.smooth()</span></code></a>).</p>
<li><p><strong>prevs</strong>– array-like of shape <cite>(n_samples, n_classes,)</cite> with the true prevalence values</p></li>
<li><p><strong>prevs_hat</strong>– array-like of shape <cite>(n_samples, n_classes,)</cite> with the predicted prevalence values</p></li>
<li><p><strong>eps</strong>– smoothing factor. KLD is not defined in cases in which the distributions contain zeros; <cite>eps</cite>
is typically set to be <spanclass="math notranslate nohighlight">\(\frac{1}{2T}\)</span>, with <spanclass="math notranslate nohighlight">\(T\)</span> the sample size. If <cite>eps=None</cite>, the sample size
will be taken from the environment variable <cite>SAMPLE_SIZE</cite> (which has thus to be set beforehand).</p></li>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.error.</span></span><spanclass="sig-name descname"><spanclass="pre">mnkld</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">prevs</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">prevs_hat</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">eps</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">None</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.error.mnkld"title="Permalink to this definition">¶</a></dt>
<dd><p>Computes the mean Normalized Kullback-Leibler divergence (see <aclass="reference internal"href="#quapy.error.nkld"title="quapy.error.nkld"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">quapy.error.nkld()</span></code></a>) across the sample pairs.
The distributions are smoothed using the <cite>eps</cite> factor (see <aclass="reference internal"href="#quapy.error.smooth"title="quapy.error.smooth"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">quapy.error.smooth()</span></code></a>).</p>
<li><p><strong>prevs</strong>– array-like of shape <cite>(n_samples, n_classes,)</cite> with the true prevalence values</p></li>
<li><p><strong>prevs_hat</strong>– array-like of shape <cite>(n_samples, n_classes,)</cite> with the predicted prevalence values</p></li>
<li><p><strong>eps</strong>– smoothing factor. NKLD is not defined in cases in which the distributions contain zeros; <cite>eps</cite>
is typically set to be <spanclass="math notranslate nohighlight">\(\frac{1}{2T}\)</span>, with <spanclass="math notranslate nohighlight">\(T\)</span> the sample size. If <cite>eps=None</cite>, the sample size
will be taken from the environment variable <cite>SAMPLE_SIZE</cite> (which has thus to be set beforehand).</p></li>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.error.</span></span><spanclass="sig-name descname"><spanclass="pre">mrae</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">p</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">p_hat</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">eps</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">None</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.error.mrae"title="Permalink to this definition">¶</a></dt>
<dd><p>Computes the mean relative absolute error (see <aclass="reference internal"href="#quapy.error.rae"title="quapy.error.rae"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">quapy.error.rae()</span></code></a>) across the sample pairs.
The distributions are smoothed using the <cite>eps</cite> factor (see <aclass="reference internal"href="#quapy.error.smooth"title="quapy.error.smooth"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">quapy.error.smooth()</span></code></a>).</p>
<li><p><strong>prevs</strong>– array-like of shape <cite>(n_samples, n_classes,)</cite> with the true prevalence values</p></li>
<li><p><strong>prevs_hat</strong>– array-like of shape <cite>(n_samples, n_classes,)</cite> with the predicted prevalence values</p></li>
<li><p><strong>eps</strong>– smoothing factor. <cite>mrae</cite> is not defined in cases in which the true distribution contains zeros; <cite>eps</cite>
is typically set to be <spanclass="math notranslate nohighlight">\(\frac{1}{2T}\)</span>, with <spanclass="math notranslate nohighlight">\(T\)</span> the sample size. If <cite>eps=None</cite>, the sample size
will be taken from the environment variable <cite>SAMPLE_SIZE</cite> (which has thus to be set beforehand).</p></li>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.error.</span></span><spanclass="sig-name descname"><spanclass="pre">mse</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">prevs</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">prevs_hat</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.error.mse"title="Permalink to this definition">¶</a></dt>
<dd><p>Computes the mean squared error (see <aclass="reference internal"href="#quapy.error.se"title="quapy.error.se"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">quapy.error.se()</span></code></a>) across the sample pairs.</p>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.error.</span></span><spanclass="sig-name descname"><spanclass="pre">nkld</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">p</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">p_hat</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">eps</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">None</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.error.nkld"title="Permalink to this definition">¶</a></dt>
<dt>Computes the Normalized Kullback-Leibler divergence between the two prevalence distributions.</dt><dd><p>Normalized Kullback-Leibler divergence between two prevalence distributions <spanclass="math notranslate nohighlight">\(p\)</span> and <spanclass="math notranslate nohighlight">\(\hat{p}\)</span>
is computed as <spanclass="math notranslate nohighlight">\(NKLD(p,\hat{p}) = 2\frac{e^{KLD(p,\hat{p})}}{e^{KLD(p,\hat{p})}+1}-1\)</span>, where
<spanclass="math notranslate nohighlight">\(\mathcal{Y}\)</span> are the classes of interest.
The distributions are smoothed using the <cite>eps</cite> factor (see <aclass="reference internal"href="#quapy.error.smooth"title="quapy.error.smooth"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">quapy.error.smooth()</span></code></a>).</p>
<li><p><strong>prevs</strong>– array-like of shape <cite>(n_classes,)</cite> with the true prevalence values</p></li>
<li><p><strong>prevs_hat</strong>– array-like of shape <cite>(n_classes,)</cite> with the predicted prevalence values</p></li>
<li><p><strong>eps</strong>– smoothing factor. NKLD is not defined in cases in which the distributions contain zeros; <cite>eps</cite>
is typically set to be <spanclass="math notranslate nohighlight">\(\frac{1}{2T}\)</span>, with <spanclass="math notranslate nohighlight">\(T\)</span> the sample size. If <cite>eps=None</cite>, the sample size
will be taken from the environment variable <cite>SAMPLE_SIZE</cite> (which has thus to be set beforehand).</p></li>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.error.</span></span><spanclass="sig-name descname"><spanclass="pre">rae</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">p</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">p_hat</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">eps</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">None</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.error.rae"title="Permalink to this definition">¶</a></dt>
<dt>Computes the absolute relative error between the two prevalence vectors.</dt><dd><p>Relative absolute error between two prevalence vectors <spanclass="math notranslate nohighlight">\(p\)</span> and <spanclass="math notranslate nohighlight">\(\hat{p}\)</span> is computed as
where <spanclass="math notranslate nohighlight">\(\mathcal{Y}\)</span> are the classes of interest.
The distributions are smoothed using the <cite>eps</cite> factor (see <aclass="reference internal"href="#quapy.error.smooth"title="quapy.error.smooth"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">quapy.error.smooth()</span></code></a>).</p>
<li><p><strong>prevs</strong>– array-like of shape <cite>(n_classes,)</cite> with the true prevalence values</p></li>
<li><p><strong>prevs_hat</strong>– array-like of shape <cite>(n_classes,)</cite> with the predicted prevalence values</p></li>
<li><p><strong>eps</strong>– smoothing factor. <cite>rae</cite> is not defined in cases in which the true distribution contains zeros; <cite>eps</cite>
is typically set to be <spanclass="math notranslate nohighlight">\(\frac{1}{2T}\)</span>, with <spanclass="math notranslate nohighlight">\(T\)</span> the sample size. If <cite>eps=None</cite>, the sample size
will be taken from the environment variable <cite>SAMPLE_SIZE</cite> (which has thus to be set beforehand).</p></li>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.error.</span></span><spanclass="sig-name descname"><spanclass="pre">relative_absolute_error</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">p</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">p_hat</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">eps</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">None</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.error.relative_absolute_error"title="Permalink to this definition">¶</a></dt>
<dt>Computes the absolute relative error between the two prevalence vectors.</dt><dd><p>Relative absolute error between two prevalence vectors <spanclass="math notranslate nohighlight">\(p\)</span> and <spanclass="math notranslate nohighlight">\(\hat{p}\)</span> is computed as
where <spanclass="math notranslate nohighlight">\(\mathcal{Y}\)</span> are the classes of interest.
The distributions are smoothed using the <cite>eps</cite> factor (see <aclass="reference internal"href="#quapy.error.smooth"title="quapy.error.smooth"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">quapy.error.smooth()</span></code></a>).</p>
<li><p><strong>prevs</strong>– array-like of shape <cite>(n_classes,)</cite> with the true prevalence values</p></li>
<li><p><strong>prevs_hat</strong>– array-like of shape <cite>(n_classes,)</cite> with the predicted prevalence values</p></li>
<li><p><strong>eps</strong>– smoothing factor. <cite>rae</cite> is not defined in cases in which the true distribution contains zeros; <cite>eps</cite>
is typically set to be <spanclass="math notranslate nohighlight">\(\frac{1}{2T}\)</span>, with <spanclass="math notranslate nohighlight">\(T\)</span> the sample size. If <cite>eps=None</cite>, the sample size
will be taken from the environment variable <cite>SAMPLE_SIZE</cite> (which has thus to be set beforehand).</p></li>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.error.</span></span><spanclass="sig-name descname"><spanclass="pre">se</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">p</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">p_hat</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.error.se"title="Permalink to this definition">¶</a></dt>
<dt>Computes the squared error between the two prevalence vectors.</dt><dd><p>Squared error between two prevalence vectors <spanclass="math notranslate nohighlight">\(p\)</span> and <spanclass="math notranslate nohighlight">\(\hat{p}\)</span> is computed as
<spanclass="math notranslate nohighlight">\(SE(p,\hat{p})=\frac{1}{|\mathcal{Y}|}\sum_{y\in \mathcal{Y}}(\hat{p}(y)-p(y))^2\)</span>, where
<spanclass="math notranslate nohighlight">\(\mathcal{Y}\)</span> are the classes of interest.</p>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.error.</span></span><spanclass="sig-name descname"><spanclass="pre">smooth</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">prevs</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">eps</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.error.smooth"title="Permalink to this definition">¶</a></dt>
<dd><p>Smooths a prevalence distribution with <spanclass="math notranslate nohighlight">\(\epsilon\)</span> (<cite>eps</cite>) as:
<spanclass="sig-name descname"><spanclass="pre">prevalence_grid</span></span><spanclass="sig-paren">(</span><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.protocol.APP.prevalence_grid"title="Permalink to this definition">¶</a></dt>
<dd><p>Generates vectors of prevalence values from an exhaustive grid of prevalence values. The
number of prevalence values explored for each dimension depends on <cite>n_prevalences</cite>, so that, if, for example,
<cite>n_prevalences=11</cite> then the prevalence values of the grid are taken from [0, 0.1, 0.2, …, 0.9, 1]. Only
valid prevalence distributions are returned, i.e., vectors of prevalence values that sum up to 1. For each
valid vector of prevalence values, <cite>repeat</cite> copies are returned. The vector of prevalence values can be
implicit (by setting <cite>return_constrained_dim=False</cite>), meaning that the last dimension (which is constrained
to 1 - sum of the rest) is not returned (note that, quite obviously, in this case the vector does not sum up to
1). Note that this method is deterministic, i.e., there is no random sampling anywhere.</p>
<spanclass="sig-name descname"><spanclass="pre">sample</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">index</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.protocol.APP.sample"title="Permalink to this definition">¶</a></dt>
<spanclass="sig-name descname"><spanclass="pre">samples_parameters</span></span><spanclass="sig-paren">(</span><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.protocol.APP.samples_parameters"title="Permalink to this definition">¶</a></dt>
<spanclass="sig-name descname"><spanclass="pre">total</span></span><spanclass="sig-paren">(</span><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.protocol.APP.total"title="Permalink to this definition">¶</a></dt>
<emclass="property"><spanclass="pre">class</span><spanclass="w"></span></em><spanclass="sig-prename descclassname"><spanclass="pre">quapy.protocol.</span></span><spanclass="sig-name descname"><spanclass="pre">AbstractProtocol</span></span><aclass="headerlink"href="#quapy.protocol.AbstractProtocol"title="Permalink to this definition">¶</a></dt>
<spanclass="sig-name descname"><spanclass="pre">total</span></span><spanclass="sig-paren">(</span><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.protocol.AbstractProtocol.total"title="Permalink to this definition">¶</a></dt>
<dd><p>Indicates the total number of samples that the protocol generates.</p>
<emclass="property"><spanclass="pre">class</span><spanclass="w"></span></em><spanclass="sig-prename descclassname"><spanclass="pre">quapy.protocol.</span></span><spanclass="sig-name descname"><spanclass="pre">AbstractStochasticSeededProtocol</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">random_state</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">0</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.protocol.AbstractStochasticSeededProtocol"title="Permalink to this definition">¶</a></dt>
<p>An <cite>AbstractStochasticSeededProtocol</cite> is a protocol that generates, via any random procedure (e.g.,
via random sampling), sequences of <aclass="reference internal"href="quapy.data.html#quapy.data.base.LabelledCollection"title="quapy.data.base.LabelledCollection"><codeclass="xref py py-class docutils literal notranslate"><spanclass="pre">quapy.data.base.LabelledCollection</span></code></a> samples.
The protocol abstraction enforces
the object to be instantiated using a seed, so that the sequence can be fully replicated.
In order to make this functionality possible, the classes extending this abstraction need to
implement only two functions, <aclass="reference internal"href="#quapy.protocol.AbstractStochasticSeededProtocol.samples_parameters"title="quapy.protocol.AbstractStochasticSeededProtocol.samples_parameters"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">samples_parameters()</span></code></a> which generates all the parameters
needed for extracting the samples, and <aclass="reference internal"href="#quapy.protocol.AbstractStochasticSeededProtocol.sample"title="quapy.protocol.AbstractStochasticSeededProtocol.sample"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">sample()</span></code></a> that, given some parameters as input,
<spanclass="sig-name descname"><spanclass="pre">collator</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">sample</span></span></em>, <emclass="sig-param"><spanclass="o"><spanclass="pre">*</span></span><spanclass="n"><spanclass="pre">args</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.protocol.AbstractStochasticSeededProtocol.collator"title="Permalink to this definition">¶</a></dt>
<emclass="property"><spanclass="pre">property</span><spanclass="w"></span></em><spanclass="sig-name descname"><spanclass="pre">random_state</span></span><aclass="headerlink"href="#quapy.protocol.AbstractStochasticSeededProtocol.random_state"title="Permalink to this definition">¶</a></dt>
<emclass="property"><spanclass="pre">abstract</span><spanclass="w"></span></em><spanclass="sig-name descname"><spanclass="pre">sample</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">params</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.protocol.AbstractStochasticSeededProtocol.sample"title="Permalink to this definition">¶</a></dt>
<dd><p>Extract one sample determined by the given parameters</p>
<emclass="property"><spanclass="pre">abstract</span><spanclass="w"></span></em><spanclass="sig-name descname"><spanclass="pre">samples_parameters</span></span><spanclass="sig-paren">(</span><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.protocol.AbstractStochasticSeededProtocol.samples_parameters"title="Permalink to this definition">¶</a></dt>
<dd><p>This function has to return all the necessary parameters to replicate the samples</p>
<li><p><strong>domainA</strong>– one domain, an object of <codeclass="xref py py-class docutils literal notranslate"><spanclass="pre">qp.data.LabelledCollection</span></code></p></li>
<li><p><strong>domainB</strong>– another domain, an object of <codeclass="xref py py-class docutils literal notranslate"><spanclass="pre">qp.data.LabelledCollection</span></code></p></li>
<li><p><strong>sample_size</strong>– integer, the number of instances in each sample; if None (default) then it is taken from
qp.environ[“SAMPLE_SIZE”]. If this is not set, a ValueError exception is raised.</p></li>
<li><p><strong>repeats</strong>– int, number of samples to draw for every mixture rate</p></li>
<spanclass="sig-name descname"><spanclass="pre">sample</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">indexes</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.protocol.DomainMixer.sample"title="Permalink to this definition">¶</a></dt>
<spanclass="sig-name descname"><spanclass="pre">samples_parameters</span></span><spanclass="sig-paren">(</span><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.protocol.DomainMixer.samples_parameters"title="Permalink to this definition">¶</a></dt>
<spanclass="sig-name descname"><spanclass="pre">total</span></span><spanclass="sig-paren">(</span><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.protocol.DomainMixer.total"title="Permalink to this definition">¶</a></dt>
<spanclass="sig-name descname"><spanclass="pre">sample</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">index</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.protocol.NPP.sample"title="Permalink to this definition">¶</a></dt>
<spanclass="sig-name descname"><spanclass="pre">samples_parameters</span></span><spanclass="sig-paren">(</span><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.protocol.NPP.samples_parameters"title="Permalink to this definition">¶</a></dt>
<spanclass="sig-name descname"><spanclass="pre">total</span></span><spanclass="sig-paren">(</span><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.protocol.NPP.total"title="Permalink to this definition">¶</a></dt>
<emclass="property"><spanclass="pre">class</span><spanclass="w"></span></em><spanclass="sig-prename descclassname"><spanclass="pre">quapy.protocol.</span></span><spanclass="sig-name descname"><spanclass="pre">OnLabelledCollectionProtocol</span></span><aclass="headerlink"href="#quapy.protocol.OnLabelledCollectionProtocol"title="Permalink to this definition">¶</a></dt>
<p>Protocols that generate samples from a <codeclass="xref py py-class docutils literal notranslate"><spanclass="pre">qp.data.LabelledCollection</span></code> object.</p>
<spanclass="sig-name descname"><spanclass="pre">RETURN_TYPES</span></span><emclass="property"><spanclass="w"></span><spanclass="p"><spanclass="pre">=</span></span><spanclass="w"></span><spanclass="pre">['sample_prev',</span><spanclass="pre">'labelled_collection']</span></em><aclass="headerlink"href="#quapy.protocol.OnLabelledCollectionProtocol.RETURN_TYPES"title="Permalink to this definition">¶</a></dt>
<emclass="property"><spanclass="pre">classmethod</span><spanclass="w"></span></em><spanclass="sig-name descname"><spanclass="pre">get_collator</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">return_type</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">'sample_prev'</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.protocol.OnLabelledCollectionProtocol.get_collator"title="Permalink to this definition">¶</a></dt>
<spanclass="sig-name descname"><spanclass="pre">get_labelled_collection</span></span><spanclass="sig-paren">(</span><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.protocol.OnLabelledCollectionProtocol.get_labelled_collection"title="Permalink to this definition">¶</a></dt>
<ddclass="field-odd"><p>an object of type <codeclass="xref py py-class docutils literal notranslate"><spanclass="pre">qp.data.LabelledCollection</span></code></p>
<spanclass="sig-name descname"><spanclass="pre">on_preclassified_instances</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">pre_classifications</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">in_place</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">False</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.protocol.OnLabelledCollectionProtocol.on_preclassified_instances"title="Permalink to this definition">¶</a></dt>
<dd><p>Returns a copy of this protocol that acts on a modified version of the original
<codeclass="xref py py-class docutils literal notranslate"><spanclass="pre">qp.data.LabelledCollection</span></code> in which the original instances have been replaced
with the outputs of a classifier for each instance. (This is convenient for speeding-up
the evaluation procedures for many samples, by pre-classifying the instances in advance.)</p>
<p>A variant of <aclass="reference internal"href="#quapy.protocol.APP"title="quapy.protocol.APP"><codeclass="xref py py-class docutils literal notranslate"><spanclass="pre">APP</span></code></a> that, instead of using a grid of equidistant prevalence values,
relies on the Kraemer algorithm for sampling unit (k-1)-simplex uniformly at random, with
k the number of classes. This protocol covers the entire range of prevalence values in a
statistical sense, i.e., unlike APP there is no guarantee that it is covered precisely
equally for all classes, but it is preferred in cases in which the number of possible
combinations of the grid values of APP makes this endeavour intractable.</p>
<spanclass="sig-name descname"><spanclass="pre">sample</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">index</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.protocol.USimplexPP.sample"title="Permalink to this definition">¶</a></dt>
<spanclass="sig-name descname"><spanclass="pre">samples_parameters</span></span><spanclass="sig-paren">(</span><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.protocol.USimplexPP.samples_parameters"title="Permalink to this definition">¶</a></dt>
<spanclass="sig-name descname"><spanclass="pre">total</span></span><spanclass="sig-paren">(</span><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.protocol.USimplexPP.total"title="Permalink to this definition">¶</a></dt>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.functional.</span></span><spanclass="sig-name descname"><spanclass="pre">HellingerDistance</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">P</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">Q</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.functional.HellingerDistance"title="Permalink to this definition">¶</a></dt>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.functional.</span></span><spanclass="sig-name descname"><spanclass="pre">TopsoeDistance</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">P</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">Q</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">epsilon</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">1e-20</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.functional.TopsoeDistance"title="Permalink to this definition">¶</a></dt>
<dd><p>Topsoe distance between two (discretized) distributions <cite>P</cite> and <cite>Q</cite>.
The Topsoe distance for two discrete distributions of <cite>k</cite> bins is defined as:</p>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.functional.</span></span><spanclass="sig-name descname"><spanclass="pre">adjusted_quantification</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">prevalence_estim</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">tpr</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">fpr</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">clip</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">True</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.functional.adjusted_quantification"title="Permalink to this definition">¶</a></dt>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.functional.</span></span><spanclass="sig-name descname"><spanclass="pre">check_prevalence_vector</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">p</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">raise_exception</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">False</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">toleranze</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">1e-08</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.functional.check_prevalence_vector"title="Permalink to this definition">¶</a></dt>
<dd><p>Checks that p is a valid prevalence vector, i.e., that it contains values in [0,1] and that the values sum up to 1.</p>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.functional.</span></span><spanclass="sig-name descname"><spanclass="pre">get_nprevpoints_approximation</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">combinations_budget</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="w"></span><spanclass="n"><spanclass="pre">int</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">n_classes</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="w"></span><spanclass="n"><spanclass="pre">int</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">n_repeats</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="w"></span><spanclass="n"><spanclass="pre">int</span></span><spanclass="w"></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="w"></span><spanclass="default_value"><spanclass="pre">1</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.functional.get_nprevpoints_approximation"title="Permalink to this definition">¶</a></dt>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.functional.</span></span><spanclass="sig-name descname"><spanclass="pre">normalize_prevalence</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">prevalences</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.functional.normalize_prevalence"title="Permalink to this definition">¶</a></dt>
<ddclass="field-odd"><p><strong>prevalences</strong>– array-like of shape <cite>(n_classes,)</cite> or of shape <cite>(n_samples, n_classes,)</cite> with prevalence values</p>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.functional.</span></span><spanclass="sig-name descname"><spanclass="pre">num_prevalence_combinations</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">n_prevpoints</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="w"></span><spanclass="n"><spanclass="pre">int</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">n_classes</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="w"></span><spanclass="n"><spanclass="pre">int</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">n_repeats</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="w"></span><spanclass="n"><spanclass="pre">int</span></span><spanclass="w"></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="w"></span><spanclass="default_value"><spanclass="pre">1</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.functional.num_prevalence_combinations"title="Permalink to this definition">¶</a></dt>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.functional.</span></span><spanclass="sig-name descname"><spanclass="pre">prevalence_from_labels</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">labels</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">classes</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.functional.prevalence_from_labels"title="Permalink to this definition">¶</a></dt>
<dd><p>Computed the prevalence values from a vector of labels.</p>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.functional.</span></span><spanclass="sig-name descname"><spanclass="pre">prevalence_from_probabilities</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">posteriors</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">binarize</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="w"></span><spanclass="n"><spanclass="pre">bool</span></span><spanclass="w"></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="w"></span><spanclass="default_value"><spanclass="pre">False</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.functional.prevalence_from_probabilities"title="Permalink to this definition">¶</a></dt>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.functional.</span></span><spanclass="sig-name descname"><spanclass="pre">prevalence_linspace</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">n_prevalences</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">21</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">repeats</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">1</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">smooth_limits_epsilon</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">0.01</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.functional.prevalence_linspace"title="Permalink to this definition">¶</a></dt>
<dd><p>Produces an array of uniformly separated values of prevalence.
By default, produces an array of 21 prevalence values, with
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.functional.</span></span><spanclass="sig-name descname"><spanclass="pre">strprev</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">prevalences</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">prec</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">3</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.functional.strprev"title="Permalink to this definition">¶</a></dt>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.functional.</span></span><spanclass="sig-name descname"><spanclass="pre">uniform_prevalence_sampling</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">n_classes</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">size</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">1</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.functional.uniform_prevalence_sampling"title="Permalink to this definition">¶</a></dt>
<ddclass="field-even"><p><cite>np.ndarray</cite> of shape <cite>(size, n_classes,)</cite> if <cite>size>1</cite>, or of shape <cite>(n_classes,)</cite> otherwise</p>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.functional.</span></span><spanclass="sig-name descname"><spanclass="pre">uniform_simplex_sampling</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">n_classes</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">size</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">1</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.functional.uniform_simplex_sampling"title="Permalink to this definition">¶</a></dt>
<ddclass="field-even"><p><cite>np.ndarray</cite> of shape <cite>(size, n_classes,)</cite> if <cite>size>1</cite>, or of shape <cite>(n_classes,)</cite> otherwise</p>
<spanid="quapy-model-selection"></span><h2>quapy.model_selection<aclass="headerlink"href="#module-quapy.model_selection"title="Permalink to this heading">¶</a></h2>
<li><p><strong>model</strong> (<aclass="reference internal"href="quapy.method.html#quapy.method.base.BaseQuantifier"title="quapy.method.base.BaseQuantifier"><em>BaseQuantifier</em></a>) – the quantifier to optimize</p></li>
<li><p><strong>param_grid</strong>– a dictionary with keys the parameter names and values the list of values to explore</p></li>
<spanclass="sig-name descname"><spanclass="pre">best_model</span></span><spanclass="sig-paren">(</span><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.model_selection.GridSearchQ.best_model"title="Permalink to this definition">¶</a></dt>
<dd><p>Returns the best model found after calling the <aclass="reference internal"href="#quapy.model_selection.GridSearchQ.fit"title="quapy.model_selection.GridSearchQ.fit"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">fit()</span></code></a> method, i.e., the one trained on the combination
of hyper-parameters that minimized the error function.</p>
<spanclass="sig-name descname"><spanclass="pre">fit</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">training</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="w"></span><spanclass="n"><aclass="reference internal"href="quapy.data.html#quapy.data.base.LabelledCollection"title="quapy.data.base.LabelledCollection"><spanclass="pre">LabelledCollection</span></a></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.model_selection.GridSearchQ.fit"title="Permalink to this definition">¶</a></dt>
<spanclass="sig-name descname"><spanclass="pre">get_params</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">deep</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">True</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.model_selection.GridSearchQ.get_params"title="Permalink to this definition">¶</a></dt>
<dd><p>Returns the dictionary of hyper-parameters to explore (<cite>param_grid</cite>)</p>
<spanclass="sig-name descname"><spanclass="pre">quantify</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">instances</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.model_selection.GridSearchQ.quantify"title="Permalink to this definition">¶</a></dt>
<dd><p>Estimate class prevalence values using the best model found after calling the <aclass="reference internal"href="#quapy.model_selection.GridSearchQ.fit"title="quapy.model_selection.GridSearchQ.fit"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">fit()</span></code></a> method.</p>
<spanclass="sig-name descname"><spanclass="pre">set_params</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="o"><spanclass="pre">**</span></span><spanclass="n"><spanclass="pre">parameters</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.model_selection.GridSearchQ.set_params"title="Permalink to this definition">¶</a></dt>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.model_selection.</span></span><spanclass="sig-name descname"><spanclass="pre">cross_val_predict</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">quantifier</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="w"></span><spanclass="n"><aclass="reference internal"href="quapy.method.html#quapy.method.base.BaseQuantifier"title="quapy.method.base.BaseQuantifier"><spanclass="pre">BaseQuantifier</span></a></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">data</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="w"></span><spanclass="n"><aclass="reference internal"href="quapy.data.html#quapy.data.base.LabelledCollection"title="quapy.data.base.LabelledCollection"><spanclass="pre">LabelledCollection</span></a></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">nfolds</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">3</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">random_state</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">0</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.model_selection.cross_val_predict"title="Permalink to this definition">¶</a></dt>
<dd><p>Akin to <aclass="reference external"href="https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.cross_val_predict.html">scikit-learn’s cross_val_predict</a>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.plot.</span></span><spanclass="sig-name descname"><spanclass="pre">binary_diagonal</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">method_names</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">true_prevs</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">estim_prevs</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">pos_class</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">1</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">title</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">None</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">show_std</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">True</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">legend</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">True</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">train_prev</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">None</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">savepath</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">None</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">method_order</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">None</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.plot.binary_diagonal"title="Permalink to this definition">¶</a></dt>
<dd><p>The diagonal plot displays the predicted prevalence values (along the y-axis) as a function of the true prevalence
values (along the x-axis). The optimal quantifier is described by the diagonal (0,0)-(1,1) of the plot (hence the
name). It is convenient for binary quantification problems, though it can be used for multiclass problems by
indicating which class is to be taken as the positive class. (For multiclass quantification problems, other plots
like the <aclass="reference internal"href="#quapy.plot.error_by_drift"title="quapy.plot.error_by_drift"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">error_by_drift()</span></code></a> might be preferable though).</p>
<dd><p>Plots the error (along the x-axis, as measured in terms of <cite>error_name</cite>) as a function of the train-test shift
(along the y-axis, as measured in terms of <aclass="reference internal"href="#quapy.error.ae"title="quapy.error.ae"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">quapy.error.ae()</span></code></a>). This plot is useful especially for multiclass
problems, in which “diagonal plots” may be cumbersone, and in order to gain understanding about how methods
fare in different regions of the prior probability shift spectrum (e.g., in the low-shift regime vs. in the
<li><p><strong>method_names</strong>– array-like with the method names for each experiment</p></li>
<li><p><strong>true_prevs</strong>– array-like with the true prevalence values (each being a ndarray with n_classes components) for
each experiment</p></li>
<li><p><strong>estim_prevs</strong>– array-like with the estimated prevalence values (each being a ndarray with n_classes components)
for each experiment</p></li>
<li><p><strong>tr_prevs</strong>– training prevalence of each experiment</p></li>
<li><p><strong>n_bins</strong>– number of bins in which the y-axis is to be divided (default is 20)</p></li>
<li><p><strong>error_name</strong>– a string representing the name of an error function (as defined in <cite>quapy.error</cite>, default is “ae”)</p></li>
<li><p><strong>show_std</strong>– whether or not to show standard deviations as color bands (default is False)</p></li>
<li><p><strong>show_density</strong>– whether or not to display the distribution of experiments for each bin (default is True)</p></li>
<emclass="property"><spanclass="pre">class</span><spanclass="w"></span></em><spanclass="sig-prename descclassname"><spanclass="pre">quapy.util.</span></span><spanclass="sig-name descname"><spanclass="pre">EarlyStop</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">patience</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">lower_is_better</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">True</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.util.EarlyStop"title="Permalink to this definition">¶</a></dt>
<spanclass="gp">>>></span><spanclass="n">earlystop</span><spanclass="o">.</span><spanclass="n">IMPROVED</span><spanclass="c1"># is True</span>
<spanclass="gp">>>></span><spanclass="n">earlystop</span><spanclass="o">.</span><spanclass="n">STOP</span><spanclass="c1"># is False (patience=1)</span>
<spanclass="gp">>>></span><spanclass="n">earlystop</span><spanclass="o">.</span><spanclass="n">STOP</span><spanclass="c1"># is True (patience=0)</span>
<spanclass="gp">>>></span><spanclass="n">earlystop</span><spanclass="o">.</span><spanclass="n">best_epoch</span><spanclass="c1"># is 1</span>
<spanclass="gp">>>></span><spanclass="n">earlystop</span><spanclass="o">.</span><spanclass="n">best_score</span><spanclass="c1"># is 0.7</span>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.util.</span></span><spanclass="sig-name descname"><spanclass="pre">create_if_not_exist</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">path</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.util.create_if_not_exist"title="Permalink to this definition">¶</a></dt>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.util.</span></span><spanclass="sig-name descname"><spanclass="pre">create_parent_dir</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">path</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.util.create_parent_dir"title="Permalink to this definition">¶</a></dt>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.util.</span></span><spanclass="sig-name descname"><spanclass="pre">download_file</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">url</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">archive_filename</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.util.download_file"title="Permalink to this definition">¶</a></dt>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.util.</span></span><spanclass="sig-name descname"><spanclass="pre">download_file_if_not_exists</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">url</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">archive_filename</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.util.download_file_if_not_exists"title="Permalink to this definition">¶</a></dt>
<dd><p>Dowloads a function (using <aclass="reference internal"href="#quapy.util.download_file"title="quapy.util.download_file"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">download_file()</span></code></a>) if the file does not exist.</p>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.util.</span></span><spanclass="sig-name descname"><spanclass="pre">get_quapy_home</span></span><spanclass="sig-paren">(</span><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.util.get_quapy_home"title="Permalink to this definition">¶</a></dt>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.util.</span></span><spanclass="sig-name descname"><spanclass="pre">map_parallel</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">func</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">args</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">n_jobs</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.util.map_parallel"title="Permalink to this definition">¶</a></dt>
<dd><p>Applies func to n_jobs slices of args. E.g., if args is an array of 99 items and <ahref="#id2"><spanclass="problematic"id="id3">`</span></a>n_jobs`=2, then
func is applied in two parallel processes to args[0:50] and to args[50:99]. func is a function
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.util.</span></span><spanclass="sig-name descname"><spanclass="pre">parallel</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">func</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">args</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">n_jobs</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">seed</span></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="default_value"><spanclass="pre">None</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.util.parallel"title="Permalink to this definition">¶</a></dt>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.util.</span></span><spanclass="sig-name descname"><spanclass="pre">pickled_resource</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">pickle_path</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="w"></span><spanclass="n"><spanclass="pre">str</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">generation_func</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="w"></span><spanclass="n"><spanclass="pre">callable</span></span></em>, <emclass="sig-param"><spanclass="o"><spanclass="pre">*</span></span><spanclass="n"><spanclass="pre">args</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.util.pickled_resource"title="Permalink to this definition">¶</a></dt>
<dd><p>Allows for fast reuse of resources that are generated only once by calling generation_func(*args). The next times
this function is invoked, it loads the pickled resource. Example:</p>
<divclass="doctest highlight-default notranslate"><divclass="highlight"><pre><span></span><spanclass="gp">>>></span><spanclass="k">def</span><spanclass="nf">some_array</span><spanclass="p">(</span><spanclass="n">n</span><spanclass="p">):</span><spanclass="c1"># a mock resource created with one parameter (`n`)</span>
<spanclass="gp">>>></span><spanclass="n">pickled_resource</span><spanclass="p">(</span><spanclass="s1">'./my_array.pkl'</span><spanclass="p">,</span><spanclass="n">some_array</span><spanclass="p">,</span><spanclass="mi">10</span><spanclass="p">)</span><spanclass="c1"># the resource does not exist: it is created by calling some_array(10)</span>
<spanclass="gp">>>></span><spanclass="n">pickled_resource</span><spanclass="p">(</span><spanclass="s1">'./my_array.pkl'</span><spanclass="p">,</span><spanclass="n">some_array</span><spanclass="p">,</span><spanclass="mi">10</span><spanclass="p">)</span><spanclass="c1"># the resource exists; it is loaded from './my_array.pkl'</span>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.util.</span></span><spanclass="sig-name descname"><spanclass="pre">save_text_file</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">path</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">text</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.util.save_text_file"title="Permalink to this definition">¶</a></dt>
<spanclass="sig-prename descclassname"><spanclass="pre">quapy.util.</span></span><spanclass="sig-name descname"><spanclass="pre">temp_seed</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">random_state</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#quapy.util.temp_seed"title="Permalink to this definition">¶</a></dt>
