Let
P
{\displaystyle P}
be a simple polygon or a rectifiable simple closed curve, and let
X
{\displaystyle X}
be any set enclosed by
P
{\displaystyle P}
.
A geodesic between two points in
P
{\displaystyle P}
is a shortest path connecting those two points that stays entirely within
P
{\displaystyle P}
.
A subset
K
{\displaystyle K}
of the points inside
P
{\displaystyle P}
is said to be relatively convex, geodesically convex, or
P
{\displaystyle P}
-convex if, for every two points of
K
{\displaystyle K}
, the geodesic between them in
P
{\displaystyle P}
stays within
K
{\displaystyle K}
. Then the relative convex hull of
X
{\displaystyle X}
can be defined as the intersection of all relatively convex sets containing
X
{\displaystyle X}
.
Toussaint (1986), who provided an efficient algorithm for the construction of the relative convex hull for finite sets of points inside a simple polygon. With subsequent improvements in the time bounds for two subroutines, finding shortest paths between query points in a polygon, and polygon triangulation, this algorithm takes time
O
(
p
+
n
log
(
p
+
n
)
)
{\displaystyle O(p+n\log(p+n))}
on an input with
n
{\displaystyle n}
points in a polygon with
p
{\displaystyle p}
vertices. It can also be maintained dynamically in sublinear time per update.
For relative convex hulls of simple polygons, an alternative but equivalent definition of convexity can be used. A simple polygon
P
{\displaystyle P}
within another simple polygon
Q
{\displaystyle Q}
is relatively convex or
Q
{\displaystyle Q}
-convex if every line segment contained in
Q
{\displaystyle Q}
that connects two points of
P
{\displaystyle P}
lies within
P
{\displaystyle P}
. The relative convex hull of a simple polygon
P
{\displaystyle P}
within
Q
{\displaystyle Q}
can be defined as the intersection of all
Q
{\displaystyle Q}
-convex polygons that contain
P
{\displaystyle P}
, as the smallest
Q
{\displaystyle Q}
-convex polygon that contains
P
{\displaystyle P}
, or as the minimum-perimeter simple polygon that contains
P
{\displaystyle P}
and is contained by
Q
{\displaystyle Q}
.
A similar definition can also be given for the relative convex hull of two disjoint simple polygons. This type of hull can be used in algorithms for testing whether the two polygons can be separated into disjoint halfplanes by a continuous linear motion, and in data structures for collision detection of moving polygons.
The definition of relative convex hulls based on minimum enclosure does not extend to higher dimensions, because (even without being surrounded by an outer shape) the minimum surface area enclosure of a non-convex set is not generally convex. However, for the relative convex hull of a connected set within another set, a similar definition to one for simple polygons can be used. In this case, a relatively convex set can again be defined as a subset of the given outer set that contains all line segments in the outer set between pairs of its points. The relative convex hull can be defined as the intersection of all relatively convex sets that contain the inner set.
Klette, Gisela (November 2010), "A recursive algorithm for calculating the relative convex hull", 25th International Conference of Image and Vision Computing New Zealand, IEEE, pp. 1–7, doi:10.1109/ivcnz.2010.6148857, ISBN 978-1-4244-9631-0, S2CID 17854252 978-1-4244-9631-0
Sklansky, Jack; Chazin, Robert L.; Hansen, Bruce J. (1972), "Minimum-perimeter polygons of digitized silhouettes", IEEE Transactions on Computers, C-21 (3): 260–268, doi:10.1109/tc.1972.5008948, S2CID 6818423 /wiki/Doi_(identifier)
Toussaint, Godfried (1986), "An optimal algorithm for computing the relative convex hull of a set of points in a polygon", Proceedings of EURASIP, Signal Processing III: Theories and Applications, Part 2, North-Holland, pp. 853–856 /wiki/Godfried_Toussaint
Guibas, Leonidas J.; Hershberger, John (1989), "Optimal shortest path queries in a simple polygon", Journal of Computer and System Sciences, 39 (2): 126–152, doi:10.1016/0022-0000(89)90041-X, MR 1024124 /wiki/Leonidas_J._Guibas
Chazelle, Bernard (1991), "Triangulating a simple polygon in linear time", Discrete & Computational Geometry, 6 (3): 485–524, doi:10.1007/BF02574703 /wiki/Bernard_Chazelle
Guibas, Leonidas J.; Hershberger, John (1989), "Optimal shortest path queries in a simple polygon", Journal of Computer and System Sciences, 39 (2): 126–152, doi:10.1016/0022-0000(89)90041-X, MR 1024124 /wiki/Leonidas_J._Guibas
Oh, Eunjin; Ahn, Hee-Kap (2017), "Dynamic geodesic convex hulls in dynamic simple polygons", 33rd International Symposium on Computational Geometry (SoCG 2017), LIPIcs, vol. 77, Schloss Dagstuhl, pp. 51:1–51:15, doi:10.4230/LIPIcs.SoCG.2017.51, MR 3685723 /wiki/Doi_(identifier)
Klette, Gisela (November 2010), "A recursive algorithm for calculating the relative convex hull", 25th International Conference of Image and Vision Computing New Zealand, IEEE, pp. 1–7, doi:10.1109/ivcnz.2010.6148857, ISBN 978-1-4244-9631-0, S2CID 17854252 978-1-4244-9631-0
Klette, Gisela (November 2010), "A recursive algorithm for calculating the relative convex hull", 25th International Conference of Image and Vision Computing New Zealand, IEEE, pp. 1–7, doi:10.1109/ivcnz.2010.6148857, ISBN 978-1-4244-9631-0, S2CID 17854252 978-1-4244-9631-0
Williams, Jason; Rossignac, Jarek (2005), "Tightening: curvature-limiting morphological simplification", in Kobbelt, Leif; Shapiro, Vadim (eds.), Proceedings of the Tenth ACM Symposium on Solid and Physical Modeling 2005, Cambridge, Massachusetts, USA, June 13-15, 2005, ACM, pp. 107–112, doi:10.1145/1060244.1060257, hdl:1853/3736, S2CID 15514388 /wiki/Doi_(identifier)
Toussaint, Godfried (1989), "On separating two simple polygons by a single translation", Discrete & Computational Geometry, 4 (3): 265–278, doi:10.1007/BF02187729, MR 0988749 /wiki/Godfried_Toussaint
Basch, Julien; Erickson, Jeff; Guibas, Leonidas J.; Hershberger, John; Zhang, Li (2004), "Kinetic collision detection between two simple polygons", Computational Geometry, 27 (3): 211–235, doi:10.1016/j.comgeo.2003.11.001, MR 2039172, S2CID 300999 /wiki/Leonidas_J._Guibas
Williams, Jason; Rossignac, Jarek (2005), "Tightening: curvature-limiting morphological simplification", in Kobbelt, Leif; Shapiro, Vadim (eds.), Proceedings of the Tenth ACM Symposium on Solid and Physical Modeling 2005, Cambridge, Massachusetts, USA, June 13-15, 2005, ACM, pp. 107–112, doi:10.1145/1060244.1060257, hdl:1853/3736, S2CID 15514388 /wiki/Doi_(identifier)
Sloboda, Fridrich; Za'ko, Bedrich (2001), "On approximation of Jordan surfaces in 3d", in Bertrand, G.; Imiya, A.; Klette, R. (eds.), Digital and Image Geometry: Advanced Lectures, Lecture Notes in Computer Science, vol. 2243, Springer, pp. 365–386, doi:10.1007/3-540-45576-0_22, ISBN 978-3-540-43079-7 978-3-540-43079-7