Advanced Topics#

Most use cases of DetecTree only make use of the TrainingSelector, ClassifierTrainer and Classifier classes and their respective methods. Nevertheless, See the “background” example notebook and the article of Yang et al. [3] for more information.

Train/test split#

In order to enhance the robustness of the classifier, it is important that the subset of pixels selected as training samples are representative of the whole dataset. Given the large variety of scenes that can be found in such a dataset of urban aerial imagery (e.g., lakes, buildings, parks, forests…), a random selection of training tiles might not be representative of such variety and therefore lead to a classifier with low overall accuracy.

To overcome such problem, Yang et al. [3] proposed a procedure of selecting training samples that intends to find the set of tiles that is most representative of the dataset. The scene structure of an image can be represented by a Gist descriptor [2], a low dimensional vector encoding which captures the high-level semantics of real-world aerial images. Following the approach of Yang et al. [3], the image descriptor is computed by:

  • convolving it with Gabor filters on 3 frequencies and 4, 8 and orientations respectively, which accounts for 320 components

  • computing a 8x8x8 joint color histogram in the Lab color space, which accounts for 512 components the two components are normalized to unit L-1 norm separately and then concatenated to form a 832-component image descriptor.

Nevertheless, the way in which such image descriptor is computer can be customized by means of the arguments of TrainingSelector.__init__. Such arguments will then be forwarded to the following function in order to compute the GIST descriptor of the input image:

detectree.image_descriptor.compute_image_descriptor_from_filepath(img_filepath, kernels, response_bins_per_axis, num_color_bins)[source]#

Compute a GIST descriptor for RGB image file.

See the background example notebook for more details.

Parameters:

  • img_filepath (str, file object or pathlib.Path object) – Path to a file, URI, file object opened in binary (‘rb’) mode, or a Path object representing the image for which a GIST descriptor will be computed. The value will be passed to rasterio.open.

  • kernels (list-like) – List of kernel 2-D arrays that correspond to the filter bank.

  • response_bins_per_axis (int) – Number of spatial bins per axis into which the responses to the filter bank will be aggregated. For example, a value of 2 will aggregate the responses into the four quadrants of the image (i.e., 2x2, 2 bins in each axis of the image).

  • num_color_bins (int) – Number of color bins per axis of the L*a*b color space with which the joint color histogram will be computed.

Returns:

img_descr – Vector representing GIST descriptor of img_rgb.

Return type:

array-like

The GIST descriptor might also be directly computed from an array with the RGB representation of the image:

detectree.image_descriptor.compute_image_descriptor(img_rgb, kernels, response_bins_per_axis, num_color_bins)[source]#

Compute a GIST descriptor for an RGB image array.

See the background example notebook for more details.

Parameters:

  • img_rgb (array-like) – The image in RGB format, i.e., in a 3-D array.

  • kernels (list-like) – List of kernel 2-D arrays that correspond to the filter bank.

  • response_bins_per_axis (int) – Number of spatial bins per axis into which the responses to the filter bank will be aggregated. For example, a value of 2 will aggregate the responses into the four quadrants of the image (i.e., 2x2, 2 bins in each axis of the image).

  • num_color_bins (int) – Number of color bins per axis of the L*a*b color space with which the joint color histogram will be computed.

Returns:

img_descr – Vector representing GIST descriptor of img_rgb.

Return type:

array-like

On the other hand, in order to obtain a Gabor filter bank (e.g., for the kernels argument), the following function can be used:

detectree.filters.get_gabor_filter_bank(frequencies, num_orientations)[source]#

Get a Gabor filter bank with different frequencies and orientations.

Parameters:

  • frequencies (list-like) – Set of frequencies used to build the Gabor filter bank.

  • num_orientations (int or list-like) – Number of orientations used to build the Gabor filter bank. If an integer is provided, the corresponding number of orientations will be used for each scale (determined by gabor_frequencies). If a tuple is provided, each element will determine the number of orientations that must be used at its matching scale (determined by gabor_frequencies) - thus the tuple must match the length of frequencies.

Returns:

kernels – List of kernel 2-D arrays that correspond to the filter bank.

Return type:

list-like

Pixel classification#

In order to perform a binary pixel-level classification of tree/non-tree pixels, each pixel is transformed into a feature vector. In DetecTree, the way in which feature vectors are computed can be customized by means of the arguments of Classifier.__init__. With the default argument values, which follow the methods of Yang et al. [3], each pixel is transformed into a 27-feature vector where 6, 18 and 3 features capture characteristics of color, texture and entropy respectively. Such arguments are forwarded to the following class:

class detectree.pixel_features.PixelFeaturesBuilder(*, sigmas=None, num_orientations=None, neighborhood=None, min_neighborhood_range=None, num_neighborhoods=None)[source]#

Customize how pixel features are computed.

__init__(*, sigmas=None, num_orientations=None, neighborhood=None, min_neighborhood_range=None, num_neighborhoods=None)[source]#

Initialize the pixel feature builder.

See the background example notebook for more details.

Parameters:

  • sigmas (list-like, optional) – The list of scale parameters (sigmas) to build the Gaussian filter bank that will be used to compute the pixel-level features. The provided argument will be passed to the initialization method of the PixelFeaturesBuilder class. If no value is provided, the value set in settings.GAUSS_SIGMAS is used.

  • num_orientations (int, optional) – The number of equally-distributed orientations to build the Gaussian filter bank that will be used to compute the pixel-level features. The provided argument will be passed to the initialization method of the PixelFeaturesBuilder class. If no value is provided, the value set in settings.GAUSS_NUM_ORIENTATIONS is used.

  • neighborhood (array-like, optional) – The base neighborhood structure that will be used to compute the entropy features. The provided argument will be passed to the initialization method of the PixelFeaturesBuilder class. If no value is provided, a square with a side size of 2 * min_neighborhood_range + 1 is used.

  • min_neighborhood_range (int, optional) – The range (i.e., the square radius) of the smallest neighborhood window that will be used to compute the entropy features. The provided argument will be passed to the initialization method of the PixelFeaturesBuilder class. If no value is provided, the value set in settings.ENTROPY_MIN_NEIGHBORHOOD_RANGE is used.

  • num_neighborhoods (int, optional) – The number of neighborhood windows (whose size follows a geometric progression starting at min_neighborhood_range) that will be used to compute the entropy features. The provided argument will be passed to the initialization method of the PixelFeaturesBuilder class. If no value is provided, the value set in settings.ENTROPY_NUM_NEIGHBORHOODS is used.

build_features(*, split_df=None, img_filepaths=None, img_dir=None, img_filename_pattern=None, method=None, img_cluster=None)[source]#

Build the pixel feature array for a list of images.

Parameters:

  • split_df (pd.DataFrame) – Data frame with the train/test split.

  • img_filepaths (list of image file paths, optional) – List of images to be transformed into features. Alternatively, the same information can be provided by means of the img_dir and img_filename_pattern keyword arguments. Ignored if providing split_df.

  • img_dir (str representing path to a directory, optional) – Path to the directory where the images whose filename matches img_filename_pattern are to be located. Ignored if split_df or img_filepaths is provided.

  • img_filename_pattern (str representing a file-name pattern, optional) – Filename pattern to be matched in order to obtain the list of images. If no value is provided, the value set in settings.IMG_FILENAME_PATTERN is used. Ignored if split_df or img_filepaths is provided.

  • method ({'cluster-I', 'cluster-II'}, optional) – Method used in the train/test split

  • img_cluster (int, optional) – The label of the cluster of images. Only used if method is ‘cluster-II’.

Returns:

X – Array with the pixel features.

Return type:

numpy ndarray

The texture features are obtained by convolving the images with a filter bank, which is obtained by means of the following function:

detectree.filters.get_texture_kernel(sigma)[source]#

Get a texture kernel based on Yang et al. (2009).

Parameters:

sigma (numeric) – Scale parameter to build a texture kernel, based on a Gaussian on the X dimension and a second-derivative Gaussian in the Y dimension

Returns:

texture_kernel

Return type:

array-like

The arguments of Classifier.__init__ also serve to customize how the pixel response (i.e., tree/non-tree labels of each pixel) is computed, by forwarding them to the following class:

class detectree.pixel_response.PixelResponseBuilder(*, tree_val=None, nontree_val=None)[source]#

Customize how pixel responses (tree/non-tree labels) are computed.

__init__(*, tree_val=None, nontree_val=None)[source]#

Initialize the pixel response builder.

See the background example notebook for more details.

Parameters:

  • tree_val (int, optional) – The values that designate tree and non-tree pixels respectively in the response images. If no values are provided, the values set in settings.TREE_VAL and settings.NON_TREE_VAL are respectively used.

  • nontree_val (int, optional) – The values that designate tree and non-tree pixels respectively in the response images. If no values are provided, the values set in settings.TREE_VAL and settings.NON_TREE_VAL are respectively used.

build_response(*, split_df=None, response_img_dir=None, response_img_filepaths=None, img_filename_pattern=None, method=None, img_cluster=None)[source]#

Build the pixel response (flat) array for a list of images.

Parameters:

  • split_df (pd.DataFrame) – Data frame with the train/test split.

  • response_img_dir (str representing path to a directory, optional) – Path to the directory where the response images are located. Required if providing split_df. Otherwise response_img_dir might either be ignored if providing response_img_filepaths, or be used as the directory where the images whose filename matches img_filename_pattern are to be located.

  • response_img_filepaths (list of image file paths, optional) – List of images to be transformed into the response. Alternatively, the same information can be provided by means of the img_dir and img_filename_pattern keyword arguments. Ignored if providing split_df.

  • img_filename_pattern (str representing a file-name pattern, optional) – Filename pattern to be matched in order to obtain the list of images. If no value is provided, the value set in settings.IMG_FILENAME_PATTERN is used. Ignored if split_df or img_filepaths is provided.

  • method ({'cluster-I', 'cluster-II'}, optional) – Method used in the train/test split.

  • img_cluster (int, optional) – The label of the cluster of images. Only used if method is ‘cluster-II’.

Returns:

responses – Array with the pixel responses.

Return type:

numpy ndarray

Optionally, the predicted pixel labels can be refined with the graph-cut max-flow procedure from Boykov and Komogrov [1] (used by default when refinement is enabled in Classifier), implemented as:

detectree.refine.maxflow_refine(p_tree_img, tree_val, nontree_val, *, refine_int_rescale=10000, refine_beta=50)[source]#

Refine the pixel-level classification using a graph max-flow algorithm.

Parameters:

  • p_tree_img (numpy.ndarray) – The probability image of the pixel being a tree, as a two-dimensional numpy array with values between 0 and 1.

  • tree_val (int, optional) – The values that designate tree and non-tree pixels respectively in the output array.

  • nontree_val (int, optional) – The values that designate tree and non-tree pixels respectively in the output array.

  • refine_int_rescale (int, optional) – Parameter of the refinement procedure that controls the precision of the transformation of float to integer edge weights, required for the employed graph cuts algorithm. Larger values lead to greater precision.

  • refine_beta (int, optional) – Parameter of the refinement procedure that controls the smoothness of the labelling. Larger values lead to smoother shapes.

Returns:

img – The refined pixel-level classification as a two-dimensional numpy array with the same shape as p_tree_img.

Return type:

numpy.ndarray

Evaluation#

You can evaluate the pixel classification using the compute_eval_metrics function, whose metrics argument accepts either metric names from sklearn.metrics (strings) or callables that accept y_true and y_pred:

detectree.evaluate.compute_eval_metrics(*, pred_img_filepaths=None, metrics=None, metrics_kwargs=None, clf=None, clf_dict=None, hf_hub_repo_id=None, hf_hub_clf_filename=None, hf_hub_download_kwargs=None, skops_trusted=None, refine_method=None, refine_kwargs=None, split_df=None, img_dir=None, response_img_dir=None, img_filepaths=None, response_img_filepaths=None, img_filename_pattern=None, **classifier_kwargs)[source]#

Compute evaluation metrics for the validation images.

Parameters:

  • pred_img_filepaths (list-like, optional) – List of paths to precomputed predicted images. If provided, classification is skipped and metrics are computed directly from these files. Only predictions with a matching response image (by basename) are used. Requires response_img_dir or response_img_filepaths.

  • metrics (str, func or list of str or func) – The metrics to compute, must be either a string with a function of the sklearn.metrics, a function that takes a y_true and y_pred positional arguments with the true and predicted labels respectively or a list-like of any of the two options. If no value is provided, the values set in settings.EVAL_METRICS are used.

  • metrics_kwargs (dict or list of dict) – Additional keyword arguments to pass to each of the metric functions.

  • clf (scikit-learn-like classifier, optional) – Trained classifier. If no value is provided, the classifier is loaded from HuggingFace Hub using the values provided in hf_hub_repo_id and hf_hub_clf_filename.

  • clf_dict (dictionary, optional) – Dictionary mapping a trained scikit-learn-like classifier to each first-level cluster label.

  • hf_hub_repo_id (str, optional) – HuggingFace Hub repository id (string with the user or organization and repository name separated by a /) and file name of the skops classifier respectively. If no value is provided, the values set in settings.HF_HUB_REPO_ID and settings.HF_HUB_CLF_FILENAME Ignored if clf or clf_dict are provided.

  • hf_hub_clf_filename (str, optional) – HuggingFace Hub repository id (string with the user or organization and repository name separated by a /) and file name of the skops classifier respectively. If no value is provided, the values set in settings.HF_HUB_REPO_ID and settings.HF_HUB_CLF_FILENAME Ignored if clf or clf_dict are provided.

  • hf_hub_download_kwargs (dict, optional) – Additional keyword arguments (besides “repo_id”, “filename”, “library_name” and “library_version”) to pass to huggingface_hub.hf_hub_download.

  • skops_trusted (list, optional) – List of trusted object types to load the classifier from HuggingFace Hub, passed to skops.io.load. If no value is provided, the value from settings.SKOPS_TRUSTED is used. Ignored if clf or clf_dict are provided.

  • refine_method (callable or bool, optional) – Method to refine the pixel-level classification. If False is provided, no refinement is performed. If None is provided, the default behavior of detectree.classifier.Classifier is used.

  • refine_kwargs (dict, optional) – Keyword arguments that will be passed to refine_method. Ignored if no refinement is performed.

  • split_df (pandas DataFrame, optional) – Data frame with the validation images.

  • img_dir (str representing path to a directory, optional) – Path to the directory where the images from split_df are located. Required if split_df is provided. Ignored if img_filepaths is provided.

  • response_img_dir (str representing path to a directory, optional) – Path to the directory where the response tiles are located. Ignored if providing response_img_filepaths. Only images with a matching response (by basename) are evaluated.

  • img_filepaths (list-like, optional) – List of paths to the tiles that will be used for validation. Ignored if split_df is provided.

  • response_img_filepaths (list-like, optional) – List of paths to the binary response tiles that will be used for evaluation. Ignored if split_df is provided. Only images with a matching response (by basename) are evaluated.

  • img_filename_pattern (str representing a file-name pattern, optional) – Filename pattern to be matched in order to obtain the list of images. If no value is provided, the value set in settings.IMG_FILENAME_PATTERN is used. Ignored if split_df or img_filepaths is provided.

  • classifier_kwargs (dict, optional) – Additional keyword arguments to pass to the initialization of detectree.classifier.Classifier class.

Returns:

metric_dict – Values of the metrics computed for the validation images. If only one metric is provided, a single value is returned. If multiple metrics are provided, a dict with a key for each metric is returned. The metric values can be of different types depending on the metric function used, e.g., precision_score returns a single float value, precision_recall_curve returns a tuple of arrays, and confusion_matrix returns a two-dimensional array.

Return type:

numeric, dict

Additionally, you can use the metrics_kwargs argument to provide per-metric options (as a list of keyword arguments passed to each matching item of metrics). The function will return the values of the metrics computed for the validation images.

It is also possible to compare the performance of refinement parameters with the eval_refine_params function, which given a refine_method (by default the graph-cut max-flow procedure of detectree.maxflow_refine as defined in the settings.CLF_REFINE_METHOD) will compute the metrics for a list of parameter dicts (refine_params_list):

detectree.evaluate.eval_refine_params(*, refine_method=None, refine_params_list=None, metrics=None, metrics_kwargs=None, clf=None, clf_dict=None, hf_hub_repo_id=None, hf_hub_clf_filename=None, hf_hub_download_kwargs=None, skops_trusted=None, tree_val=None, nontree_val=None, split_df=None, img_dir=None, img_filepaths=None, img_filename_pattern=None, response_img_dir=None, **classifier_kwargs)[source]#

Evaluate a refinement procedure for different parameters.

Parameters:

  • refine_method (callable, optional) – Refinement method that takes a probability image as the first positional argument followed by tree and non-tree values, e.g., refine_method(p_tree_img, tree_val, nontree_val, **kwargs). If no value is provided, the value from settings.CLF_REFINE_METHOD is used.

  • refine_params_list (list of dict, optional) – Parameters to evaluate for the refinement method, as a list of keyword arguments. The metrics will be computed for each item of this list. If no value is provided, the value from settings.EVAL_REFINE_PARAMS is used.

  • metrics (str, func or list of str or func) – The metrics to compute, must be either a string with a function of the sklearn.metrics, a function that takes a y_true and y_pred positional arguments with the true and predicted labels respectively or a list-like of any of the two options. If no value is provided, the values set in settings.EVAL_METRICS are used.

  • metrics_kwargs (dict or list of dict) – Additional keyword arguments to pass to each of the metric functions.

  • clf (scikit-learn-like classifier, optional) – Trained classifier. If no value is provided, the classifier is loaded from HuggingFace Hub using the values provided in hf_hub_repo_id and hf_hub_clf_filename.

  • clf_dict (dictionary, optional) – Dictionary mapping a trained scikit-learn-like classifier to each first-level cluster label.

  • hf_hub_repo_id (str, optional) – HuggingFace Hub repository id (string with the user or organization and repository name separated by a /) and file name of the skops classifier respectively. If no value is provided, the values set in settings.HF_HUB_REPO_ID and settings.HF_HUB_CLF_FILENAME Ignored if clf or clf_dict are provided.

  • hf_hub_clf_filename (str, optional) – HuggingFace Hub repository id (string with the user or organization and repository name separated by a /) and file name of the skops classifier respectively. If no value is provided, the values set in settings.HF_HUB_REPO_ID and settings.HF_HUB_CLF_FILENAME Ignored if clf or clf_dict are provided.

  • hf_hub_download_kwargs (dict, optional) – Additional keyword arguments (besides “repo_id”, “filename”, “library_name” and “library_version”) to pass to huggingface_hub.hf_hub_download.

  • skops_trusted (list, optional) – List of trusted object types to load the classifier from HuggingFace Hub, passed to skops.io.load. If no value is provided, the value from settings.SKOPS_TRUSTED is used. Ignored if clf or clf_dict are provided.

  • tree_val (int, optional) – The values that designate tree and non-tree pixels respectively in the response images. If no values are provided, the values set in settings.TREE_VAL and settings.NON_TREE_VAL are respectively used.

  • nontree_val (int, optional) – The values that designate tree and non-tree pixels respectively in the response images. If no values are provided, the values set in settings.TREE_VAL and settings.NON_TREE_VAL are respectively used.

  • split_df (pandas DataFrame, optional) – Data frame with the validation images.

  • img_dir (str representing path to a directory, optional) – Path to the directory where the images from split_df are located. Required if split_df is provided. Ignored if img_filepaths is provided.

  • img_filepaths (list-like, optional) – List of paths to the tiles that will be used for validation. Ignored if split_df is provided.

  • img_filename_pattern (str representing a file-name pattern, optional) – Filename pattern to be matched in order to obtain the list of images. If no value is provided, the value set in settings.IMG_FILENAME_PATTERN is used. Ignored if split_df or img_filepaths is provided.

  • response_img_dir (str representing path to a directory, optional) – Path to the directory where the response tiles are located. Ignored if providing response_img_filepaths.

  • classifier_kwargs (dict, optional) – Additional keyword arguments to pass to the initialization of detectree.classifier.Classifier class.

Returns:

results – A DataFrame with the computed values for each metric (row) and each refinement keyword argument set (column, stringified).

Return type:

pandas DataFrame

The function returns a DataFrame with metrics as rows and parameter sets as columns.