• Definition of segmentation parameters, primarily scale.
• The distinction between single and multi-scale.
• Differentiation between supervised and unsupervised determination methods, with a focus on unsupervised methods as they are multiple.
• Overview of advanced approaches and state-of-the-art methods for determining segmentation parameters.
• Segmentation without parameter optimization.

3.1 Sub Definition of the Concepts and Categories of Segmentation Parameters Most Frequently Encountered in the Literature

For this literature review, conceptual details and methodological explanations are provided, and mathematical details are avoided for simplicity.

Segmentation is a crucial step and a foundational procedure for GEOBIA [42]. Segmentation methods and techniques must be robust to obtain reliable and representative results. And any inadequate parameterization results in erroneous results with insufficient segmentation [43], low quality, and difficulty in comparing methods [10], and the classification's accuracy will be put to the test, as its quality is tightly related to the quality of the segmented object [17] [44] and [45]. Therefore, accurate and perfect segmentation requires appropriate segmentation parameters, which is difficult compared to the complexities involved in remote sensing imagery [17]. Each segmentation method and technique has appropriate segmentation parameters. These parameters control the segmented object’s size, shape, and boundaries; however, the most widely used algorithm for segmentation is multi-resolution (MRS). Because of this, the three parameters to be specified repeatedly for segmentation are compactness, scale, and shape [41] and [46], and the most important for the segmentation step for most of the techniques is scale [4]. It is used to control intra-segment homogeneity and inter-segment heterogeneity and define the size of segments for the entire image.

Furthermore, an object's scale alludes to its spatial extent or size in the image, and the optimal object scale relates to the smallest class-optimal size or ideal size for different classes [18].

It should be noted that the determination of the scale parameter is not limited to the optimization scale. Still, it is a widespread tuning of the parameter that regulates the sizes and shapes of the image fragments. This parameter has different names depending on the segmentation algorithm and software used; for example, the multiresolution segmentation implemented in the eCognition software calls it "scale," and this is the most well-known name; the edge-based watershed algorithm implemented in the ENVI software calls it "scale level," and the BerkeleyImageSeg software calls it a threshold (Figure 4).

Figure 4: Most commonly used software for segmentation and determining optimal segmentation parameters

Currently, the process of determination for all the algorithms consists of varying the scale parameters, such as their name, and the object's size adjustment and optimization. In this sense, the method of determining the segmentation parameters, especially the scale, is transferable when using different algorithms and in diverse landscapes, and when using different kinds of remote sensing images scenes [47].

Several scale categories have emerged in the literature. Consequently, it is possible to distinguish absolute scale, which refers to individual reality objects, and relative scale, which concerns spatial data resolution [4]. Furthermore, scales can be considered as windows or a function [4]. Ming et al., [42] revealed three different levels of scale notation for RS images, namely pixel-based, object-based, and pattern-based scales. Adding to this is the so-called spatial or spectral scale; differently, most scaling methods is either spectrum or space-derived [19].

3.2 Unique Scale Versus Multi-Scale for Segmentation

As previously stated, scale is crucial in the segmentation of images into homogeneous objects. In the field of segmentation, there is a distinction between unique scale and multiscale. Indeed, some cases depend only on a unique scale for the segmentation of homogeneous areas mostly [48] and [49]. However, multiscale image segmentation is performed for optimal scale determination in most cases.

Multiscale or multiresolution image segmentation refers to the fundamental procedure and driving force of OBIA in which the pixelized image is converted into homogeneous primitive objects dissimilar to the neighboring objects [19] and [50]. In this case, the scaling parameter governs the ideal size of the image objects based on a high threshold, which the user defines during the multiscale segmentation process for allowable variation in heterogeneity. Therefore, these are either the minimum heterogeneity measures or the minimum homogeneity measures of spectral and geometric pattern features [19]. In practice, multiscale analysis requires several adequate scale parameters to assess the landscape structure at multiple organization levels [19].

Generally, using a global threshold to segment the different objects or segments is the joint action, but this produces a single segmentation, which does not allow the different geographical objects to be split into distinct and dissimilar regions [1]. This presents a real limitation of this single scale and does not provide the opportunity for some useful objects to appear [21]. This limitation can be overcome by using hierarchical image objects that achieve a better representation result [18]. Beyond that, the cross-scale strategy also merges the multiscale segmentation objects and provides complete segmentation results at adaptive scales at different land coverages [51]. Selecting multiple substantial segmentation scales has been favored if the relevant segmentation scale should vary for various geographical objects [17] [19] [52] and [53]; as for further sequential analysis, several segmentation scales ranging from coarse to fine were defined. Though, the evaluation metric cannot determine the optimal segmentation scale for diverse geographical objects, considering that it is always computed for all segments.

Figure 5: Circuit of optimal segmentation parameters methods

Moreover, additional efforts are required for identifying or combining the best segmentation scales for a sequential analysis [1]. Nevertheless, Lang and Langanke [54] have conclusively attested that in some cases, a single-level representation can suffice and be simpler; conversely, Johnson et al., [55] and several other researchers demonstrate that multiscale segmentation is the most efficient and cost-effective in terms of geo-object representativeness after comparing the two.

3.3 Supervised/Unsupervised Approach for Determining Segmentation Parameters

Following Wang et al., [17], a distinction should be made in terms of assessment methods (Figure 5) between objective analytical methods that may include quantitative and system-level analysis and standard visual inspection or visual assessment [56]. The latter relies on simple, manual, visual comparisons using a trial-and-error approach for time-consuming, laborious, and subjective manipulation, with results that vary by user and require an interesting level of expertise [30]. But to some extent, trial-and-error is acceptable [57] and widely used [21]. Conversely, methods built on quantitative assessment could be classified into two categories: supervised and unsupervised parameter selection [17] [27] [55] [56] [58] [59] [60] [61] [62] [63] and [64] have been proposed over the past decade to parameterize the MRS SP and the parameters of other segmentation algorithms [30]. Both sets of methods have advantages and disadvantages.

Supervised methods measure the similarities or overlaps between segmented objects and reference objects. This evaluation uses arithmetic or geometric divergence measures to estimate the overlap degree or difference between reference polygons and generated segments [30]. Since this method involves reference polygons, the difficulty of determining these polygons depends on experts whose opinions may differ [17]. Among the methods used, Withrana and Civco [65] used Euclidean distance measures (ED2) for divergence, Ma et al., [66] used the fitting equation, Yan et al., [11] used fuzzy logic to compute segmentation parameters, and under-segmentation and over-segmentation metrics were used by Bialas et al., [57] to decide the optimal segmentation parameters. The researchers agreed that supervised methods lack universality and are less adaptable to differences in soil characteristics and changing perspectives. Supervised scale parameter selection methods, although subjective and tedious, are currently quite widely used in segmentation due to the ease of repeated trials and lack of best choice [19].

Unsupervised or fully automatic methods, empirical goodness, or empirical mismatch methods, rely on statistical and mathematical measures to decide the quality of segmentation results [17], such as measures that optimize inter-segment heterogeneity and intra-segment homogeneity [30] by satisfying predetermined conditions. The application of these approaches does not require polygons. These methods are an objective and repeatable process [67], are efficient, fast, and less subjective since they do not depend on other reference polygons and interpreters [17] and [30]. On the other hand, they are still computationally intensive [30]. Hay et al., [68] pioneered the use of these methods. These researchers expanded on Woodcock and Strahler's [69] idea based on local variance (LV) reaching the conclusion that the spatial resolution associated with the maximum LV and optimal viewing scale was determined by the size of the objects in the scene and integrated them into the different tree objects that make up the scene, from which spectral, spatial, and optimal variance measures of each object are defined. Espindola et al., [70] determined the segmentation parameters using a global objective function based on the inherent properties of the data, namely Moran's spatial autocorrelation Index and variance.

Both values are normalized and integrated to determine the urban areas optimal segmentation parameters and a good choice of scaling parameter that corresponds to the combination of low inter-segment Moran's I with low intra-segment near-infrared (NIR) band weighted variance. In a similar study, Kim et al., [71] and [72] calculated the unweighted variance and overall Moran's index in the NIR band. They were the first to report that VL increased with PS in a semi-variable manner when they plotted them separately to compare several segments and identify the most appropriate parameters for the segmentation of forest stands. Drăgut et al., [73] presented a procedure based on the focal mean statistic to optimize segmentation scale parameters. Drăgut et al., [52] suggested that an object heterogeneity local variance (VL) in a scene could specify the relevant scale level. Thus, Scale Parameter Estimation tool (ESP) is built to identify the best multi-resolution segmentation parameters. Karl and Maurer [74] resolved appropriate segmentation scales using variogram-based prediction. Anders et al., [75] evaluated the quality of segmentation outputs for every specific geomorphic feature type to optimize segmentation parameters. Johnson and Xie [63] presented yet another multi-scale method for estimating optimal segmentation based on local and global assessments. Initially, they determined the optimal scale using a global intra-segment and inter-segment heterogeneity measure using Moran's weighted variance and index, respectively; then, the local heterogeneity measures were used to detect under- or over-segmented regions, with the aim of refining this region using the local heterogeneity statistic, which significantly improves the global segmentation results. However, NIR, red, and green spectral bands were used in this study, discovering that the NIR band was fruitful for assessment objectives. The approach presented by Zhang et al., [76] is built on the weighted sum of homogeneity within regions and heterogeneity between regions. The weight is decided by the proportions of change of the two, helping equilibrium proportions of change of the two parts; the segmentation using the smallest rate value is regarded as the optimal segmentation, but this process cannot determine the exact optimal segmentation. Ming et al., [43] achieved good results by using a classical semivariogram process to identify the segmentation scale of RS images. Analogous to the ESP tool developed by Drăgut et al., [52] and Zhao et al. [77] used the modified ROC-LV method, using the eCognition software’s multi-resolution segmentation to review the optimal scale in slope segmentation. Drăgut et al., [56] used the potential of local variance as a method based on spectral statistics to identify scale changes in the input data, the average LV of all layers' objects is approximated. If the LV value recorded by the scale level is equal to or less than the previous value, the iterations are canceled. To this end, they developed the Scale Parameter Estimation (ESP2), a data-driven unsupervised PS determination optimization technique and tool that studies the LV at different PS intervals [78]. It was improved for simultaneous application of LV on multiple image layers and automatic defining of scale parameters, performing a three-level hierarchy concept to catch image objects of distinct sizes [56]. Researchers have dissected this version to determine different segmentation scales for different regions, from fine to coarse [79]. Furthermore, using the ESP2 tool, multiple SP levels and structures of different image objects can be estimated [30]. On the contrary, in Definiens® software, is limited by the multi-resolution segmentation, which still does not solve the problem of poor segmentation and poor fusion, and only uses intra-segment homogeneity. To determine inter-object heterogeneity, Chen et al., [80] incorporate a boundary factor into the computation of the global MI. Indeed, with the aim of assessing an optimal segmentation scale, the indicator combines global variance and global MI.

Espindola’s et al., [70] method adapted from Cánovas-García and Alonso-Sarría [81], with Moran's I index replaced by Geary's index (for intrasegment heterogeneity), and including object variability when optimizing the SP to develop a local SP optimization technique. The latter could be advantageous to evaluate large study areas that cover various land use/cover types due to their local optimization nature (uniform spatial units).

Yang et al., [47] used an energy function method to improve intrasegment homogeneity calculated by the average spectral angle in the segment. In contrast, intersegment heterogeneity is computed as the weighted gradient from a segment to its neighbors using spectral angle and is incorporated into the local peak (LP) based method for automatic scaling parameterization. They confirmed that the local peak-based method using the energy function is more suitable for optimal segmentation scale setting than the mean spectral angle. Yang et al., [82] used local variance to measure object homogeneity. Johnson et al., [55] used the SP unsupervised optimization method and tool (USPO) that relies solely on image statistics to optimize MS-OBIA.

This method measures intrasegmental homogeneity using the area-weighted variance (WV) of all segments at the segmentation level; inter-segment heterogeneity uses Moran's global I as a spatial autocorrelation measure to track the spectral similarity between segments and their neighbors and joins these two metrics of over- and under-segmentation by the F-measure. Low WV indicates segmentation with high intrasegment homogeneity, and low MI shows segmentation with high intersegment heterogeneity. Echoing Zhang et al., [28] findings that the F-measure was more sensitive to over- and under-segmentation, Johnson et al., [55] discovered that using F-measure to combine the weighted variance and Moran's I measure was more efficient than adding them additively.

More recently, Hamedianfar et al., [49] merged Taguchi's robust statistical technique and Espindola et al., [70] objective function to find the SRM parameters optimal combinations for single scale classification; this merger was successfully used and yielded encouraging results.

Grybas et al., [64] compared some unsupervised approaches that evaluate multi-scale segmentation results using several statistical quality indicators [83]. These are global score or objective function (GS) [63], local variance [56], and overall quality (OG) [55]. Grybas et al., [64] demonstrate that GS and OG produce relatively similar results, while local variance performs the worst. However, GS provides a largely ignored but serious source of instability, making the quality ranking inconsistent. Böck et al., [67] attempted to mitigate these problems so that the method can be explored in the future.

3.3.1 Pre-estimation of parameters

The most remarkable thing about the unsupervised methods of segmentation parameter determination is that the common ground between these methods and applications is a posteriori estimation of multiscale segmentation rather than a prediction of optimal scale parameters [18]. They are often based on the multi-resolution algorithm, which has a very larger scope for the scale parameter and makes it challenging to comprehend the relationship between the scale selection indicators and scale parameter [18]. This justifies that the scale parameter is a poorly structured problem [78]. To overcome this sterility in pre-estimation methods, Ming et al., [18] applied spectral and geospatial statistical methods to clustering-based pattern recognition to pre-estimate the optimal scale parameter in multiscale segmentation. They used three scaling parameters, namely spatial bandwidth, spectral bandwidth, and fusion threshold using the average local variance (ALV). The selected parameters perform well, achieving high homogeneity and heterogeneity in the segments [18]. To rationalize the different structural levels of the landscape and for the multi-scale analysis, Liu et al., [19] used ALV and ROCLV for posterior methods. While Louw and van Niekerk [78] extended ALV to an object boundary local variance (OBLV), which aims to measure terrain topography to decide whether object boundaries intersect morphological discontinuities, and the local variance ratio (LVR) supposes that terrain variation along morphological boundaries will be more pronounced than that of the interiors of relief components. This is a patent relationship between LV, OBLV, and LVR but not a universal correlation between SP optimization techniques [78]. Liu et al., [19] argue that the ESP method enhances the local variance to a limited extent, and the OBLV graph supports the detection of multiple scale parameters.

Furthermore, Qiu et al., [84] proposed a method combining adaptive scale parameter pre-estimation based on spatial statistics and mean shift segmentation. In the case of land cover classification studies, the results indicate that the predicted scale parameter could ensure a classification result with high accuracy and exhaustiveness. Following the same approach, Ming et al., [85] used an adaptive pre-estimation method of optimal scale parameters based on spatial statistics for cropland extraction. The results confirmed the method’s validity and accelerated the object-oriented classification procedure. More recently, Xu et al., [86] extracted croplands from high spatial resolution RS images using stratified pre-estimation of segmentation parameters.

3.3.2 Use of heterogeneity vs. homogeneity measures

Most of the existing literature emphasizes the role of local and global spectral information to measure homogeneity. Yet, studies dismiss the contribution of heterogeneity measures [17].

A homogeneity measure can distinguish between high internal and low external homogeneity. Some criteria for homogeneity and heterogeneity measures such as spectrum, shape, size, texture, and context must be considered when analyzing complex landscapes [87]. Moreover, Louw and van Niekerk [78] consider that the spectral homogeneity of objects is governed by color criteria and the shape homogeneity employed for merging the two adjacent objects in the case of MRS, and if this homogeneity is less than the user-defined SP.

In contrast, Shen et al., [21] developed an unsupervised multi-scale method to optimize segmentation through local spectral heterogeneity estimates for HR images. This method relies on local spectral heterogeneity curves to assess the heterogeneity within and between objects, and searches for the best object from segmentation results at diverse scales by analyzing the spectral angle for an object. Nevertheless, many researchers have used both homogeneity and heterogeneity measures [17] [18] [63] [70] [71] [88] [89] [90] [91] and [92]. As a matter of fact, they used indices of homogeneity within segmentation objects or regions and heterogeneity between segmentation objects to select the best scaling parameter.

Among the methods discussed, the ratio of the mean difference to neighbors (ABS) used in standard deviation (RMAS) method [48] enables defining a unique scale. Furthermore, it investigates the shortcomings of two different methods of optimal scale selection, namely, local variance presented by Woodcock and Strahler [69] and maximum area presented by Huang [93]. This method optimizes segmentation scale by minimizing intra-class heterogeneity and maximizing inter-class heterogeneity when the RMAS is maximal. Indeed, the RMAS curve is shown to have several local peaks and that the optimal scale is comparable and generally falls within a range of values [19].

More recently, common boundary of each segment and its neighbor, as well as local spectral data and local surface features for an HSR image was used to combine a measure of homogeneity and heterogeneity [17]. The latter followed this method to produce a value of the global segmentation parameter since the optimal segmentation parameter cannot be defined through local homogeneity and heterogeneity indicators [17]. Similarly, Zhang et al., [1] define the optimal segmentation scale as the one that maximizes intra-segment homogeneity and inter-segment heterogeneity. Nevertheless, the selected optimal segmentation scale always contains too coarse or too fine segments. Thus, it is complicated for a single segmentation generated by a global scale parameter to distinguish between different geographic objects [63].

3.4 Advanced Approaches in the Field

3.4.1 Scaling by type of geographical objects / local approach

According to the examples, most optimal scaling methods for scale segmentation are based on local variance, ROCLV, VLR, AVL, variograms, and semi-variograms to determine the optimal scale or a scale range for reliable and accurate segmentation. However, there are still types of geographic objects that are unsegmented, over-segmented, or under-segmented, especially individual trees, cars, rooftops, and lakes. Faced with these situations, the trend in terms of scale parameter optimization is towards local approaches such as adaptive computation and scaling by geographic objects. Local approaches could produce more beneficial outcomes compared to global approaches. Indeed, these methods enable seizing the spectral contrast between objects, and tend to result in more appropriate SPs [81].

Early, Felzenszwalb and Huttenlocher [94] used natural image to implement object-specific optimization methods. However, its effectiveness on complex HR images is unknown. Similarly, the relevance of Akçay and Aksoy [95] optimization method that identifies optimal object-specific segmentation scales was only proved for a restricted number of multi-scale segmentations resulting from morphological operations. Yi et al., [96] emphasized that to achieving the right segmentation results for target applications, it is essential to select appropriate segmentation scales for different land objects and combine them intelligently. They started with the primary concept of dividing the whole image into several regions using the EEMW algorithm, with land objects of a comparable optimal segmentation scale, for each of the regions. Then, the synthesis of each region’s suboptimal segmentation is performed to reach the final segmentation result. This method, called scale synthesis, produces more accurate segmentation results. Zhang and Du [97] suggest the idea that instead of automatically selecting one or more optimal segmentation scales via global evaluation metric, scaling parameters that locally fit different regions, or objects should be determined. They mentioned two methods: one based on local tuning of the global optimal scale parameter, depending on the local structure heterogeneity [98] [99] [100] [101] and [102]. The latter is known as structure-specific local optimization strategy. The second approach is using the global evaluation metric after partitioning the image into different segments to obtain the optimal segmentation scale for each region [1] [103] and [104]. The segmentation performance of these methods depends on the effectiveness of the global evaluation measures employed. Moreover, the theoretical basis of the scale parameter localization strategy is worthy of further investigation to assure that the local scale parameters are optimal for every geographical object [1]. Hu et al., [105] tried simplifying the segmentation tree.

However, the method they developed does not facilitate resolving each object’s optimal segmentation. Ideally, Shen et al., [21] indicate that scaling parameters will need to adapt to different regions or land covers. Zhang et al., [1] exploited the homogeneity variation of each segment between neighboring segmentation scales to assess each geographical object’s optimal segmentation scale. Indeed, they developed a technical foundation, using a segment tree model to build integration interactions between segments at neighboring scales. Consequently, the study's findings were a minimum and maximum segmentation scale range, which serves as the unique parameter the object-specific optimization method suggested. These results show that the difference in overall segmentation accuracy between coarse and fine fusion results is insignificant, as both increase with scale. Therefore, proving the advantage of including a wide range in the proposed optimization approach to improve the segmentation performance [1].

Moreover, Wang et al., [17] used an unsupervised method for selecting segmentation parameters using the MRS algorithm for segmentation [106], spectral information, and local spatial statistics, employing intra-segment homogeneity (WSH), inter-segment heterogeneity (BSH) and F-measure that presents a compromise between the two to learn about the quality and the optimal parameter. It is also the role of the shared boundary between each segment and its neighbors, using local spatial statistics and local surface characteristics, perfecting the considered share of external heterogeneity [17]. Therefore, in the case of over-segmentation, WSH values will be high, and MI (or BSH) values will be low, and the reverse is true in the case of under-segmentation. They show that BSH is more separable than MI in segmentation results and is more sensitive to sub-segmentation, proving the advantage of considering local spatial statistics in the BSH measurements. BSH and WSH measures reveal their effectiveness in changing the segmentation quality as the segmentation scale grows. Nevertheless, these measures are enabled of defining the optimal scale [17]. Wang et al., [17] used also an unsupervised method for selecting segmentation parameters using the MRS algorithm for segmentation [106], spectral information, and local spatial statistics, drawing on both inter-segment heterogeneity (BSH), intra-segment homogeneity (WSH) measures, and F-measure that presents a compromise between the two to learn about the quality and the optimal parameter.

More recently, Wang et al. [26] used variational scale segmentation based on spectral indices and local spatial statistics (VL) for the display of various geographical objects in high-resolution RS images, through the optimization of segmentation scale. They also developed a watershed transformation based on the scalable evolutionary watershed method [26]. Yi et al., [96], which is difficult, sophisticated, and complex, the second is to choose an initial so-called optimal scale, and then refine the under-or over-segmented objects (combination of different results) [102].

In addition to the scale of geographic objects studies focus on the determination of the suitable approaches and methods by spatial scene type; For instance, Wang et al., [17] compared their method that uses BHS and WHS with four other methods to define the optimal scale parameters in urban and rural images, these are Espindola et al., [70], Johnson et al., [55], Yang et al., [82] and Zhang et al., [76]. The results show that the method of Wang et al., [17] and Espindola et al.,[70] obtains the best and most accurate overall segmentation compared to the other three-parameter optimization methods. Compared to the tree belts, the results obtained by Wang et al., [17], Espindola et al., [70] and Johnson et al., [55] are the best. However, the method presented by Zhang et al., [76] shows an over-segmentation in most cases, which oppose to Yang et al., [82] method, which offers gross under-segmentation [17]. This may result from the complexity of calculating the heterogeneity limiting spatial statistics due to the poor disparity between the geographic objects and their adjacent objects in rural images, compared to urban images [17].

The same comparison approach was applied to the test rural areas’ images. The resulting outputs outline the effectiveness of Wang et al., [17] method in determining the optimal scale for high-quality segmentation. Conversely, the method of Yang et al., [82] was the least effective in obtaining high-quality segmentations [17]. Indeed, the method did not perform as well on urban images as on rural images [17]. Compared to the forest environment, the spectral variations are larger in HSR images [17]. To improve this subject and produce more credible information, we planned future work to build a database of segmentation parameter optimization methods by scene type and remote sensing imagery concerning this theme.

3.4.2 Adaptive calculation of the scale parameter

Yang et al., [53] proposed a computation method based on an unsupervised multi-band scaling parameter for multiscale image segmentation. This method takes all spectral bands and each pair of pixels’ spectral angle in physical image patches (PIPs), estimating their spectral homogeneity.

With a gradual change of scale parameters, the spectral homogeneity of PIPs declines until they become equivalent to semantic image objects (SIOs) by conforming to a real-world object in the image. The SP that yields the smallest mean spectral angle value (equivalent to the highest spectral homogeneity) was selected as an optimal SP [30]. Chini et al., [107], also used semantic image object scale parameters in spectral disparity using PIP fusion and SOS algorithms. The algorithm improves classification accuracy by considering multiple scales of concern and each segmentation map at the same time. Liu et al., [19] used measures of heterogeneity and homogeneity as well as geometric and spectral features to determine the scale parameters for multiscale segmentation. They used semantic image objects (SIOs) consisting of adjacent homogeneous pixels representing a physical image parcel (PIP), implying the maximum possible scope of heterogeneity generated by the integration of various PIPs. They measured the spatial autocorrelation of adjacent PIPs at each scale. The results show that, in the PIP scale/autocorrelation plot, the scale with a negative or low autocorrelation represents the optimal. Conversely, a positive autocorrelation leads to an over-segmentation or an under-segmentation.

Liu et al., [20] highlighted the need for autonomously computed scaling parameters describing homogeneous and heterogeneous intervals of neighboring pixels in spatial and spectral spaces for concurrent segmentation. They claim that estimating the linear density of the vector edge could be used to approximate the complexity of SIO in an image.

They used the GSCVE algorithm to compute parameters such as spectral and spatial scales and the reference value for the minimum map unit size (MMUS). The algorithm was developed for multi-scale segmentation and has the advantage of adaptively computing the global scale parameter for image segmentation [20]. GSCVE algorithm’s effectiveness is validated on several VHR images. However, it still has some limitations of GSCVE. It provides global scaling adjustment for multiscale segmentation, while the local scale adjustment must be achieved concomitantly. To smooth the edges, one of the post-segmentation steps must be the contour improvement of SIOs composed of PIPs [20].

3.4.3 Towards a generalizable approach for determining segmentation parameters

Current automated methods used to identify appropriate segmentation scaling parameters are scene-specific and frequently require complex. Therefore, a broad estimation strategy of relevant scale parameters could significantly speed up the GEOBIA workflow [30].

Determination of appropriate and generalizable image segmentation parameters to map urban structure by Johnson and Jozdani [30] performed a meta-analysis and mathematical modelling via a nonlinear regression tree (RT), which was constructed from information collected from previous studies, which were 1) image-based information, including the spatial and radiometric resolutions of the images; 2) information on the parameters of the MRS algorithm, including the SP, shape/color, and compactness/smoothness parameters used in each study; and 3) land cover information gathered [30]. They noticed that precise SPs could be recognised using the RT modeling results for all land covers except water. Moreover, for building and vegetation RT models generated quite precise segmentation compared to the ESP tool. However, the latter performed better results for the water class, ultimately, successful segmentation of the different land cover types was possible. Ultimately, successful segmentation of diverse land cover categories required the employment of different SPs [30] and the method employed is transferable.

4. Segmentation without Parameter Optimization

An unsupervised scene-independent approach was developed by Jozdani et al., [62] to optimize the SP in the aim of extracting urban buildings of multiple sizes. Therefore, a second-order polynomial regression model was constructed, associating the appropriate SPs to the average building size and spatial resolution of the image, assuming similar common sizes of buildings in a given urban. Moreover, this analysis enabled the estimate of appropriate multiple SPs, thus accurately extracting buildings of different sizes, without resorting to intensive computation. In cases that can be described as extreme, the researchers do not go through scale optimization; as an example, Martha et al., [108] first used a small PS for initial segmentation to generate over-segmented results. Then the over-segmented outputs were introduced into the chessboard segmentation algorithm for refinement and merging. This leads to more suitable final segmentation results without directly optimizing PS. Similarly, aiming to improve segmentation results without using a vigorous trial-and-error method, Witharana and Lynch [59] performed a combination of the segmentation drawn from the multi-threshold segmentation (MTS) algorithm deducted from SRM.

Chini et al., [107] also used the semantic image object (SIO) scaling parameter in spectral disparity using PIP fusion. The method consists of a scaled object selection (SOS) algorithm [107], which analyzes the use of all hierarchical segmentation in the classification process by integrating unsupervised hierarchical segmentation with supervised per-pixel classification. This operation helps improving the classification performance by examining the different scales of interest and taking into account each segmentation map at the same time. The SOS algorithm comprises the following steps [107]: (1) unsupervised hierarchical segmentation; (2) supervised classification of each pixel (maximum likelihood); and (3) final PIP classification.

5. Evaluation Metrics

To decide the quality of the optimal segmentation parameters, evaluation metrics based on the segmentation results are used. These metrics provide information on over- and under-segmentation errors [41] and [109]. Methods to evaluate segmentation quality are manual, with time-consuming visual interpretation, subjective with reproducible results [110], and indirect application-based methods; qualitative evaluation is achieved through supervised and unsupervised, as well as other geometric and non-geometric approaches [2]. Supervised methods compare the segmentation results to a reference dataset and use divergence or convergence criteria to measure the overlapping area between the two [111]. In contrast, unsupervised approaches do not use reference data, they are limited to measuring specific characteristics of segments such as spectral homogeneity [2] and inter-object heterogeneity [100] and find a compromise between high intra-class heterogeneity and high inter-class heterogeneity [62]. The most frequently used method in the literature is the Scale Parameter Estimation (SPE) tool [56], followed by the Global Score (GS) Johnson, however, it is important to note that these methods do not indicate which types of errors dominate, under/over-segmentation, they are faster and less subjective [30].

For supervised methods, several metrics have been used in previous work to evaluate segmentation quality, namely, Purity Index (PI), Segmentation Evaluation Index (SEI), Potential Segmentation Error (PSE), number Segment ratio (NSR), bilateral overlap index, Over Segmentation Error (OSE), and Under Segmentation Error (USE), as well as Fitness function (F), Euclidean distance (ED2), D-metric, F-measure (F-score), precision-recall curves, precision and recall to measure under- segmentation and over-segmentation [1] [2] [27] [28] [51] [55] [60] [82] [89] [112] and [113]. Extensively, Costa et al., [2] provided a state-of-the-art for evaluating the accuracy and quality of image segmentation in land-cover mapping applications and examined supervised methods representing 66 metrics. Jodzani and Chen [29] compared 21 metrics to evaluate the segmentation of buildings of various forms. The most used metrics are ED2, F-measure, D-metric, OSE, and USE [2] and [29].

To clarify the supervised approach, the research by Clinton et al., [27], Räsänen et al., [114], Whiteside et al., [115], Costa et al., [2] and Jodzani and Chen [29] have compared dozens of metrics. These studies are valuable for reviewing the field of image segmentation accuracy and quality assessment. See these studies for more details

The evaluation of image segmentation accuracy is often at an early stage of relative maturity in land cover mapping applications. Information on the generated evaluation is given against extensive use of qualitative evaluation based on visual interpretation [116]. In this sense, Ye et al., [117] reviewed several papers and found that about 16% of them included a method for evaluating segmentation accuracy, with supervised methods most commonly used for segmentation quality evaluation with 38.2% of the cases examined by Zhang’s et al., [118] study, no method is dominant except for some recommendations. Deciding on an adequate method is not evident and is a complex decision that must take into consideration the advantages and disadvantages of potential methods, such as convenience and biases. However, it is important to note that there is generally no right or wrong method. The suitability of a method ultimately depends on its compatibility with the corresponding application.

6. Conclusion

Eventually, acquiring scale parameters for multiscale segmentation is frequently based on real experiments or repeated arbitrary assessments, known as trial-and-error methods, so there is a lack of global models or frameworks for computing scale parameters and the universality of methods or algorithms in this sense. First, attention is drawn to the fact that there is no ideal scale for image objects of different sizes, shapes, and spatial distributions that make up a scene, reflecting the essence of the modifiable area unit problem (MAUP). In addition, separating the various geographical objects becomes challenging when the segmentation is based on a single global scale parameter [63].