Skip to main content

The searching algorithm for detecting a Markovian target based on maximizing the discounted effort reward search

Abstract

This paper presents the searching algorithm to detect a Markovian target which moves randomly in M-cells. Our algorithm is based on maximizing the discounted effort reward search. At each fixed number of time intervals, the search effort is a random variable with a normal distribution. More than minimizing the non-detection probability of the targets at time interval i, we seek for the optimal distribution of the search effort by maximizing the discounted effort reward search. We present some special cases of one Markovian and hidden target. Experimental results for a Markovian, hidden target are obtained and compared with the cases of applying and without applying the discounted effort reward search.

Introduction

The searching problem for missing targets had begun since the fifties of the last century. Scientists have presented different types of research plans that fit the nature of the research area. The targets were placed sometimes in difficult terrain areas on the surface of the ground or in the deep of the sea. In order to increase the probability of detection or minimize the search effort, specialists in this field dived the areas to be searched in a set with identical or different states. The search area is divided into cells of different forms. Hong et al. [1, 2] divided the area into hexagonal cells. They proposed an approximation algorithm for the optimal search path. This algorithm optimizes an approximate path to compute the detection probability, by using the conditional probabilities and then finding the maximum probability of detection of this search path. Song and Teneketizs [3] determined the optimal search strategies with multiple sensors that maximize the total probability of successful search where the target is hidden in one of a finite set of different cells. Teamah et al. [4] divided the search region into square cells. They minimized the probability of undetected and the searching effort (is bounded by a normal distribution) by using multiple searchers. They studied some special cases when the target is hidden in one of M-identical cells and when the effort is unrestricted.

It is getting harder in the case of search for two related randomly moving or located targets. El-Hadidy [5] studied this interesting problem by dividing the search region into square cells. A first investigation of this new search model (discrete search model where the targets have a motion with a discrete state-time stochastic process on a discrete state space) is presented by El-Hadidy [5] to find two related Markovian targets. This model minimized the expected effort of detecting two related targets. This mathematical model allows us to include the search effort as a function with fuzzy parameter (discounted parameter) where search effort is bounded by a normal random variable. Since there is a whole uncertainty in determining the target location at any time interval, this gave him a strong justification for using the fuzzy logic. On the other hand, this uncertainty was affected on the effort distribution. Thus, his model is not only new, but also it is a first investigation that uses a fuzzy logic in the optimal search theory. He formulated a very interesting problem, that is, a fuzzy multi-objective nonlinear stochastic minimax discounted effort reward problem. This problem can be considered as a better motivation for the fuzzy extension stochastic optimization problem. The Kuhn-Tucker conditions were applied to solve it and gave the minimum expected effort to detect the Markovian targets. Furthermore, this problem was solved in the special cases of locating targets and unbounded effort. Also, he presented a dynamic programming algorithm that gives the optimal distribution of an effort which makes the discounted effort reward of finding the targets maximized. In addition, this algorithm can be considered for these special cases. The effectiveness of this model has been presented in some real-life applications. Several studies for different kinds of optimal search plans for the lost targets on the lines, in the plane, and in the space have been studied, as in El-Hadidy et al. [636].

The main contributions of this paper center around studying the M-states search problem for two related lost targets, an extension of the problem that studied in El-Hadidy [5]. The related targets either located in one of a finite set of different states or moved through them according to discrete state and time stochastic process (discrete-time Markovian targets). This situation occurs when the located targets are very important such as searching for the spider landmines (see https://www.youtube.com/watch?v=XH0n6I0qMZA) and when they are moving such as two related submarines on the ocean. The effort must be divided among the states to find the targets. This search effort at each fixed number of time intervals is a random variable that has a normal distribution. Our purpose here is to obtain the optimal distribution of effort that maximizes the discounted effort reward of finding the targets. This minimizes the probability of undetection and the cost of finding the targets.

The rest of the paper is organized as follows. The “Problem formulation” section discusses the problem and provides the optimal values of the minimum search effort and the maximum probability of detection. The “One Markovian target” section gives special cases of one Markovian and hidden target. The “Application” section presents simulation examples, with numerical results for a Markovian and hidden target. These results are compared with the cases of applying and without applying the discounted effort reward search. This comparison can be shown in the effectiveness of this solution. Finally, the “Conclusion and future research” section concludes the paper.

Problem formulation

In this section, we present the same model which has been studied before by El-Hadidy [5] but without using fuzzy logic. This model uses the same discrete approach that was used in El-Hadidy [5] where the targets move on discrete state space (M-cells) with a discrete-time Markovian motion.

The searching technique

The searcher has the ability to move freely on M-cells (the searcher can jump from any cell to another freely). The searcher will detect the primary target and then its related target which may be in one of the primary target’s neighbor cells. Since the searcher aims to find the optimal method to get the minimum distribution of the searching effort that minimizes the searching cost, we will use all the previous hypotheses to formulate a very interesting and difficult optimization problem. El-Hadidy [5] showed the probability that the primary target exists in cell j at time interval i is denoted by Pij,i=1,2,...,N,j=1,2,...,M and consequently the probability of the other target is one of the probabilities: {Pi(jh−1),Pi(jh),Pi(jh+1),Pi(j−1),Pi(j+1),Pi(j+h−1),Pi(j+h),Pi(j+h+1)}, see Fig. 1.

Fig. 1
figure1

The neighbor cells of the cell j

The searching effort

We let the effort is randomly distributed, then we can consider that the effort which will be distributed among the cells is L(R) and its value is bounded by a random variable X (i.e., 0≤L(R)≤X). Here, the probability of detection depends on the total amount of effort Zij, i=1,2,...,N,j=1,2,...,M is applied there by the searcher and not on the way the effort is applied. We assume that the searches at distinct time intervals are independent and the motion of the target is independent of the sensors’ actions. The searcher will visit the cell j through one of its adjacent cells as in the cases in Fig. 2.

Fig. 2
figure2

All possible paths to detect the first target in the cell j and time interval i

The probability of detection

We consider that the conditional probability of detecting the target at time interval i with Zij amount of effort given that the target is located in state j is given by the detection function b(i,j,Zij). El-Hadidy [5] showed that the probability of detecting the first target in the cell j at time interval i is Pij(1−b(i,j,Zij)), where Zij is the amount of effort, given that the target is located in cell j. It is known that the number of the cells which surrounds the cell where the first target is detected at the time interval i is 8, so the other target will be detected in one of these cells at the same time. We must not forget that the searcher entered one of these eight cells before the detection of the first target. Therefore, we have seven cells and the probability of the other target will be distributed on them, see Hong et al. [1]. Here, the searcher does not enter the cells that he entered before in this time interval i. Then, the searcher will enter one of the seven cells and leaving only 6 cells with the target being distributed. Consequently, the probability of detecting the other target is \(\Psi _{ij}=6\sum _{\varpi }P_{i(j+\varpi)}\left (1-b\left (i,j,Z_{i(j+\varpi)}\right)\right),\)ϖ=−h−1,−h,−h+1,−1,1,h−1,h,h+1. For further clarification, see El-Hadidy [5]. Here, we will deal with the probability of undetecting the two targets in the cell j at time interval i which is given by Pijb(i,j,Zij)+Ψij where \(\Psi _{ij}=6\sum {\varpi }P_{i(j+\varpi)}\left (b\left (i,j,Z_{i(j+\varpi)}\right)\right),\)ϖ=−h−1,−h,−h+1,−1,1,h−1,h,h+1. Consequently, the probability of undetecting the two targets over the whole time is given by,

$$\begin{aligned} H(Z)&=\left[\left(P_{11}b(1,1,Z_{11})+\Psi_{11}\right) +\left(P_{12}b(1,2,Z_{12})+\Psi_{12}\right)\right. \\& \left. +...+\left(P_{1M}b(1,M,Z_{1M})+\Psi_{1M}\right) \right]\\ &\times \left[ \left(P_{21}b(2,1,Z_{21})+\Psi_{21}\right) +\left(P_{22}b(2,2,Z_{22})+\Psi_{22}\right) +...\right.\\&\quad \left.+\left(P_{2M}b(2,M,Z_{2M})+\Psi_{2M}\right) \right] \\ &\times...\\ & \times \left[ \left(P_{N1}b(N,1,Z_{N1})+\Psi_{N1}\right) +\left(P_{N2}b(N,2,Z_{N2})+\Psi_{N2}\right)\right. \\& \left.+...+\left(P_{NM}b(N,M,Z_{NM})+\Psi_{NM}\right) \right], \end{aligned} $$

and it can be written as,

$$ H\left(Z\right)=\prod\limits_{i=1}^{N}\sum\limits_{j=1}^{M}\left[P_{ij}b\left(i,j,Z_{ij}\right)+\Psi_{ij}\right]. $$
(1)

And the total effort of detecting the two targets is,

$$ L(Z)=\sum\limits_{j=1}^{M}\sum\limits_{i=1}^{N}\left[ Z_{ij}+\sum\limits_{\varpi }Z_{i(j+\varpi)}\right], $$
(2)

where \(\sum _{\varpi }Z_{i(j+\varpi)},\)ϖ=−h−1,−h,−h+1,−1,1,h−1,h,h+1 is the effort to detect the other target.

The exponential detection function

In physics, the signal detector is based on an exponential function because the detection exponential function has much lower computational complexity than the others such as the Gaussian kernelized energy detector, see Luo et al. [37]. Thus, here in order to model the effort, we use an exponential detection function, that is, \(1-b\left (i,j,Z_{ij}\right)=1-e^{-(Z_{ij}/T_{j})}\) and \( 1-b\left (i,j,Z_{i(j+\varpi)}\right)=1-e^{-(Z_{i(j+\varpi)}/T_{j+\varpi })},\)ϖ=−h−1,−h,−h+1,−1,1,h−1,h,h+1, where Tj and Tj+ϖ are factors due to the searching process (which depending on the nature of the cells and its dimensions) in the cell j and its neighbors, respectively. Then, the probability of undetecting the targets over the whole time is given by,

$$ H(Z)=\prod\limits_{i=1}^{N}\sum\limits_{j=1}^{M}\left[ P_{ij}e^{-(Z_{ij}/T_{j})}+\Psi_{ij}\right], $$
(3)

where \(\Psi _{ij}=6\sum \limits _{\varpi }P_{i(j+\varpi)}e^{-\left (Z_{i(j+\varpi)}/T_{j+\varpi }\right)},\)ϖ=−h−1,−h,−h+1,−1,1,h−1,h,h+1.

Optimization problem with discounted effort reward

As in El-Hadidy [5] and Blum et al. [38], we use an exponential function \( w_{j}(i)=\lambda _{j}^{i},0<\lambda _{j}<1\) that will reduce the possible rewards at time interval i. The tuning parameter λj permits us to decide indirectly how fast we want to find the targets or in other words how important are the actions that the searcher will take in the future. Here, we need to minimize the probability of undetected; then, we use the complement function of wj(i), that is, \(1-\lambda _{j}^{i}.\) The cost function (3) is combined with the discounted effort function to develop the final discounted effort reward function:

$$ H(Z;\lambda)=\prod\limits_{i=1}^{N}\sum\limits_{j=1}^{M}\left[\left(1-\lambda_{j}^{i}\right)P_{ij}e^{-\left(Z_{ij}/T_{j}\right)}+\Psi_{ij}\right], $$
(4)

where \(\Psi _{ij}=6\sum \limits _{\varpi }\left (1-\lambda _{j+\varpi }^{i}\right)P_{i(j+\varpi)}e^{-\left (Z_{i(j+\varpi)}/T_{j+\varpi }\right)}\) and the unrestricted effort will become,

$$ L\left(Z;\lambda\right)=\sum\limits_{i=1}^{N}L_{i}(Z)=\sum\limits_{j=1}^{M}\sum\limits_{i=1}^{N} \left[\left(1-\lambda_{j}^{i}\right)Z_{ij}+\Omega_{ij}\right] \leq \sum\limits_{i=1}^{N}X_{i}=X, $$
(5)

where \(\Omega _{ij}={\sum \nolimits }_{\varpi }\left (1-\lambda _{j+\varpi }^{i}\right)Z_{i(j+\varpi)}.\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \)

Let X be a random variable with a normal distribution. It has a probability density function f(x) and distribution function F(x). The purpose here is to minimize Zij,Zi(j+ϖ),λj and λj+ϖ, and thus, we have different types of decision variables and parameters in the objective function. This leads us to consider our problem as a multi-objective nonlinear programming problem aims to minimize H(Z;λ) subject to the constraints: L(Z;λ)≤X, Zij≥0, Ωij>0 and \(\sum \limits _{j=1}^{M}\left (P_{ij}+\sum _{\varpi }P_{i(j+\varpi)}\right) =1,\) where Z is a function on X. Since the detection function is exponential, then the problem will become a convex nonlinear programming problem (NLP) as follows,

NLP:

$$\begin{aligned} \underset{Z_{ij},Z_{i(j+\varpi)},\lambda_{j},\lambda_{j+\varpi}}{\min} \ & H(Z;\lambda)=\prod\nolimits_{i=1}^{N}{\sum\nolimits}_{j=1}^{M} \left[\left(1-\lambda_{j}^{i}\right) P_{ij}e^{-(Z_{ij}/T_{j})}+\right.\\ &\left. 6{\sum\nolimits}_{\varpi }\left(1-\lambda_{j+\varpi}^{i}\right)P_{i(j+\varpi)}\left(e^{-(Z_{i(j+\varpi)}/T_{j+\varpi })}\right)\right],\\ \textit{sub. to} \quad & {Z}\left({X}\right) \mathit{=}\left(Z\in R^{NM}\mid L_{i}(Z;\lambda) \leq Z\left(X_{i}\right), \right. \\ &\left. L(Z;\lambda)={\sum\nolimits}_{i=1}^{N}{\sum\nolimits}_{j=1}^{M}\left[ \left(1-\lambda_{j}^{i}\right)Z_{ij}+{\sum\nolimits}_{\varpi }\left(1-\lambda_{j+\varpi}^{i}\right)Z_{i(j+\varpi)}\right]\right. \\ &\quad \left.\leq {\sum\nolimits}_{i=1}^{N}L_{i}(Z;\lambda)=X\right), \\ &Z_{ij}\geq 0, Z_{i(j+\varpi)}\geq 0, 0<\lambda_{j}<1, 0<\lambda_{j+\varpi }<1, \\ &{\sum\nolimits}_{j=1}^{M}\left(P_{j}+{\sum\nolimits}_{\varpi }P_{j+\varpi }\right) =1 \textit{\ }\forall \textit{\ }i=1,2,...,N, \\ &\varpi =-h-1,-h,-h+1,-1,1,h-1,h,h+1\ \ \text{and} \ j=1,2,...,M\ \ . \\ \end{aligned} $$

where RNM is the feasible set of constrained decisions. The unique solution is guaranteed by the convexity of H(Z;λ) and Z(X).

Since we have two kinds of probabilities: (1) the probability of the target in each cell and (2) the probability of detecting the target, the carrying out of the search space (M- different states) with the greatest possible probability ≤1 will save the time and the effort. Hence, the detection probability (objective function) will be affected by the constraint \({\sum \nolimits }_{j=1}^{M}\left (P_{j}+\sum _{\varpi }P_{j+\varpi }\right) =1\). In addition, the targets jump between the cells with transition Markov matrix (stochastic matrix). Thus, at each time interval i, there exists a transition probability from state j (or j+ϖ) to another state, that is, Pij(orPi(j+ϖ)), this probability is computing from the stochastic matrix (see the “Application” section). This leads us to consider Pij(orPi(j+ϖ)) that is not a given parameter but a constraint where its maximum and minimum values effect directly on Zij,Zi(j+ϖ),λj and λj+ϖ. This probability is used in the formulation of the objective function; then, we call our problem as nonlinear stochastic programming problem. One can think Zij,Zi(j+ϖ) have the same type of decision variables although they used on different cells. Here, each cell has a different nature from the other so the searching methods (search devices used and etc.) differs from the cell to other. Beside that, we consider that the probability of detection in state j (or j+ϖ) at time interval i depends only on the total amount of effort applied there by the searcher and not on the way the effort is applied. Thus, we consider Zij,Zi(j+ϖ) are the effort different variables.

Definition 1

\(\bar {Z}\in Z\left (X\right) \) is said to be an optimal solution for problem (NLP) if ZZ(X) does not exist such that \( H(Z;\lambda)\leq H\left (\bar {Z};\overset {-}{\lambda }\right)\) with at least one strict inequality holds, with probability P(Li(Z;λ)≤X)≤β, β[0,1].

Now, we have the corresponding nonlinear stochastic programming problem (NLSP) as,

NLSP:

$$\begin{aligned} \underset{Z_{ij},Z_{i(j+\varpi)},\lambda_{j},\lambda_{j+\varpi }}{\min } \mathit{\ }\ & H(Z;\lambda)=\prod\nolimits_{i=1}^{N}{\sum\nolimits}_{j=1}^{M} \left[\left(1-\lambda_{j}^{i}\right)\mid P_{ij}e^{-(Z_{ij}/T_{j})} \right.\\ &\quad \left.+6{\sum\nolimits}_{\varpi }\left(1-\lambda_{j+\varpi}^{i}\right)P_{i(j+\varpi)}\left(e^{-(Z_{i(j+\varpi)}/T_{j+\varpi })}\right)\right], \\ \textit{sub. to} \qquad & P\left(L_{i}(Z;\lambda)\leq X_{i}\right) \leq \beta,\textit{\ } \beta \in [0,1], \\ & Z_{ij}\geq 0, Z_{i(j+\varpi)}\geq 0, 0<\lambda_{j}<1, 0<\lambda_{j+\varpi }<1\ ,\\ &{\sum\nolimits}_{j=1}^{M}\left(P_{j}+\sum\limits_{\varpi }P_{j+\varpi }\right) =1\textit{\ }\forall \textit{\ }i=1,2,...,N\ \ ,\\ &\varpi=-h-1,-h,-h+1,-1,1,h-1,h,h+1 and \ j=1,2,...,M\textit{\ . } \end{aligned} $$

The constraint \(\tilde {P}\left (L_{i}(Z;\lambda)\leq X_{i}\right) \geq 1-\beta \) has to be satisfied with its complement probability of at least (1−β) and can be restated as \(\tilde {P}\left (\frac {L_{i}(Z;\lambda)-E\left (X_{i}\right) }{\sqrt {Var\left (X_{i}\right) }}\leq \frac {X_{i}-E\left (X_{i}\right) }{\sqrt {Var\left (X_{i}\right) }}\right) \geq 1-\beta.\) Here, we consider that X has a normal distribution because one of the important advantages of the normal distribution is that they are sensitive to shifts in the searching effort at any time interval i. For the complement probability, we have \(\tilde {P}\left (\frac {L_{i}(Z;\lambda)-E\left (X_{i}\right) }{\sqrt {Var\left (X_{i}\right) }}\geq \frac {X-E\left (X_{i}\right) }{\sqrt {Var\left (X_{i}\right) }}\right) \leq \beta,\) where \( \frac {X_{i}-E\left (X_{i}\right) }{\sqrt {Var\left (X_{i}\right) }}\) is a standard normal random variable. If Kp represents the value of the standard normal random variable at which ϕ(Kp)=β, then this constraint can be expressed as \(\phi \left (\frac {L_{i}(Z;\lambda)-E\left (X_{i}\right) }{\sqrt {Var\left (X_{i}\right) }}\right) \leq \)ϕ(Kp). This inequality will be satisfied only if: \(\frac {L_{i}(Z;\lambda)-E\left (X_{i}\right) }{\sqrt {Var\left (X_{i}\right) }}\leq K_{p}\), i.e., \( L_{i}(Z;\lambda)-E\left (X_{i}\right) \leq K_{p}\sqrt {Var\left (X_{i}\right) }.\) Thus, the NLSP is equivalent to the following nonlinear stochastic programming problem (NLSP(1)),

NLSP(1):

$$\begin{aligned} &\underset{Z_{ij},Z_{i(j+\varpi)},\lambda_{j},\lambda_{j+\varpi }}{\min } H(Z;\lambda)=\prod\nolimits_{i=1}^{N}{\sum\nolimits}_{j=1}^{M}\left[\left(1-\lambda_{j}^{i}\right) P_{ij}e^{-(Z_{ij}/T_{j})}\ \right.\\&\quad \left.+6{\sum\nolimits}_{\varpi }(1-\lambda_{j+\varpi }^{i})P_{i(j+\varpi)}\left(e^{-(Z_{i(j+\varpi)}/T_{j+\varpi })}\right)\right], \\& \textit{sub. to}\ \ L_{i}(Z;\lambda)-E\left(X_{i}\right) \leq K_{p} \sqrt{Var\left(X_{i}\right)}, \\& Z_{ij}\geq 0,Z_{i(j+\varpi)}\geq 0,0<\lambda_{j}<1,0<\lambda_{j+\varpi }<1, \\& {\sum\nolimits}_{j=1}^{M}\left(P_{j}+\sum_{\varpi }P_{j+\varpi }\right) =1\textit{\ }\forall \textit{\ }i=1,2,...,N,\ \ \\& \varpi =-h-1,-h,-h+1,-1,1,h-1,h,h+1\ \text{and} \ j=1,2,...,M \textit{\ . } \end{aligned} $$

Which is equivalent to,

$$\begin{aligned} &\underset{Z_{ij},Z_{i(j+\varpi)},\lambda_{j},\lambda_{j+\varpi }}{\min} H(Z;\lambda)=\prod\nolimits_{i=1}^{N}{\sum\nolimits}_{j=1}^{M}\left[ \left(1-\lambda_{j}^{i}\right) P_{ij}e^{-\left(Z_{ij}/T_{j}\right)}\right.\\&\quad \left.+6{\sum\nolimits}_{\varpi }\left(1-\lambda_{j+\varpi }^{i}\right)P_{i(j+\varpi)}\left(e^{-(Z_{i(j+\varpi)}/T_{j+\varpi })}\right)\right],\\& \textit{sub. to \ } {Z}(X)\,=\,\left(Z\in R^{NM}\!\mid\! g\left(Z;\lambda\right)\,=\,{\sum\nolimits}_{j=1}^{M}\left[\left(1-\lambda_{j}^{i}\right)Z_{ij}\,+\,{\sum\nolimits}_{\varpi } \!\left(1\,-\,\lambda_{j+\varpi}^{i}\right)Z_{i(j+\varpi)}\right] \right. \\& \quad \left. -E\left(X_{i}\right) -K_{p}\sqrt{Var\left(X_{i}\right) }\leq 0\right), \\& Z_{ij}\geq 0,Z_{i(j+\varpi)}\geq 0,0<\lambda_{j}<1,0<\lambda_{j+\varpi }<1,\ \\& {\sum\nolimits}_{j=1}^{M}\left(P_{j}+{\sum\nolimits}_{\varpi }P_{j+\varpi }\right) =1\textit{\ }\forall \textit{\ }i=1,2,...,N,\\& \varpi =-h-1,-h,-h+1,-1,1,h-1,h,h+1 and \ j=1,2,...,M\textit{\ . } \end{aligned} $$

Maximum probability of detection with minimum effort

Since H(Z;λ) is an exponential function, then it can be easy to prove that H(Z;λ) is convex functions, and then the necessary Kuhn-Tucker conditions are obtained as in Mangasarian [39].

$$ \frac{\partial H_{K}(Z;\lambda)}{\partial Z_{\sigma \theta }} +U\sum\limits_{\sigma =1}^{N}\frac{\partial g_{\sigma }(Z;\lambda)}{ \partial Z_{\sigma \theta }}=0, \text{I} $$
(I)
$$ \frac{\partial H_{K}(Z;\lambda)}{\partial Z_{\sigma (\theta +\varpi)}} +U\sum\limits_{\sigma =1}^{N}\frac{\partial g_{\sigma }(Z;\lambda)}{ \partial Z_{\sigma (\theta +\varpi)}}=0, \text{II} $$
(II)
$$ \frac{\partial H_{K}(Z;\lambda)}{\partial \lambda_{\theta }} +U\sum\limits_{\sigma =1}^{N}\frac{\partial g_{\sigma }(Z;\lambda)}{ \partial \lambda_{\theta }}=0, \text{III} $$
(III)
$$ \frac{\partial H_{K}(Z;\lambda)}{\partial \lambda_{\theta +\varpi }} +U\sum\limits_{\sigma =1}^{N}\frac{\partial g_{\sigma }(Z;\lambda)}{ \partial \lambda_{\theta +\varpi }}=0, \text{IV} $$
(IV)
$$ g_{\sigma }(Z;\lambda)\leq 0, \text{V} $$
(V)
$$ Ug_{\sigma }(Z;\lambda)=0,U\geq 0.\qquad \text{VI} $$
(VI)

Implies to,

$$ {\begin{aligned} &-\frac{\left(1-\lambda_{\theta }^{\sigma }\right)P_{\sigma \theta }}{T_{\theta }}.e^{-\left(\frac{Z_{\sigma \theta }}{T_{\theta }}\right) }\prod\nolimits_{\substack{ i=1 \\ i\neq \sigma }}^{N}{\sum\nolimits}_{j=1}^{M}\left[ \left(1-\lambda_{j}^{i}\right) P_{ij}e^{-\left(\frac{Z_{ij}}{T_{j}}\right) }\right.\\& \left.+6{\sum\nolimits}_{\varpi }\left(1-\lambda_{j+\varpi }^{i}\right)P_{i(j+\varpi)}e^{-\left(\frac{Z_{i(j+\varpi)}}{T_{j+\varpi }}\right) }\right] + U\left(1-\lambda_{\theta }^{\sigma }\right) =0, \end{aligned}} $$
(6)
$$ {\begin{aligned} &-6{\sum\nolimits}_{\varpi }\frac{\left(1-\lambda_{\theta +\varpi }^{\sigma }\right)P_{\sigma (\theta +\varpi)}}{T_{\theta +\varpi }}.e^{-\left(\frac{ Z_{\sigma (\theta +\varpi)}}{T_{\theta +\varpi }}\right) }\prod\nolimits_{\substack{ i=1 \\ i\neq \sigma }}^{N}{\sum\nolimits}_{j=1}^{M}\left[ \left(1-\lambda_{j}^{i}\right) P_{ij}e^{-\left(\frac{Z_{ij}}{T_{j}}\right) }\right.\\&\left.+6{\sum\nolimits}_{\varpi }\left(1-\lambda_{j+\varpi }^{i}\right)P_{i(j+\varpi)}e^{\left(\frac{Z_{i(j+\varpi)}}{T_{j+\varpi }}\right) }\right]+U\left(1-\lambda_{\theta +\varpi }^{\sigma }\right)=0, \end{aligned}} $$
(7)
$$ {\begin{aligned} &-\sigma \lambda_{\theta }^{(\sigma -1)}P_{\sigma \theta }.e^{-\left(\frac{ Z_{\sigma \theta }}{T_{\theta }}\right) }\prod\nolimits_{\substack{ i=1 \\ i\neq \sigma }}^{N}{\sum\nolimits}_{j=1}^{M}\left[\left(1-\lambda_{j}^{i}\right) P_{ij}e^{-\left(\frac{Z_{ij}}{T_{j}}\right) }\right. \\&\quad \left. +6{\sum\nolimits}_{\varpi }\left(1-\lambda_{j+\varpi }^{i}\right)P_{i(j+\varpi)}e^{-\left(\frac{Z_{i(j+\varpi)}}{T_{j+\varpi }}\right) }\right]-U\sigma \lambda_{\theta }^{\sigma -1}Z_{\sigma \theta }=0, \end{aligned}} $$
(8)
$$ {\begin{aligned} &-6\sigma \sum_{\varpi }\lambda_{\theta +\varpi }^{(\sigma -1)}P_{\sigma (\theta +\varpi)}.e^{-\left(\frac{Z_{\sigma (\theta +\varpi)}}{T_{\theta +\varpi }}\right) } \prod\nolimits_{\substack{ i=1 \\ i\neq \sigma }}^{N}{\sum\nolimits}_{j=1}^{M}\left[ \left(1-\lambda_{j}^{i}\right) P_{ij}e^{-\left(\frac{Z_{ij}}{T_{j}}\right) }\right. \\& \left.+6{\sum\nolimits}_{\varpi }\left(1-\lambda_{j+\varpi }^{i}\right)P_{i(j+\varpi)}e^{-\left(\frac{Z_{i(j+\varpi)}}{T_{j+\varpi }}\right) }\right]-U\sigma \lambda_{\theta +\varpi }^{\sigma -1}Z_{\sigma (\theta +\varpi)}=0, \end{aligned}} $$
(9)
$$ U\left\{ \sum\limits_{j=1}^{M}\left[ \left(1-\lambda_{j}^{\sigma }\right)Z_{\sigma j}+\sum\limits_{\varpi }\left(1-\lambda_{j+\varpi }^{\sigma }\right)Z_{\sigma (j+\varpi)}\right] -E\left(X_{\sigma }\right) -K_{p}\sqrt{Var\left(X_{\sigma }\right) }\right\} =0, $$
(10)

where, \(-Z_{\sigma \theta }\leq 0,^{{}}-Z_{\sigma (\theta +\varpi)}\leq 0,^{{}}\lambda _{j}-1<0,^{{}}\lambda _{j+\varpi }-1<0,\ {\sum \nolimits }_{j=1}^{M}\left (P_{j}+\sum _{\varpi }P_{j+\varpi }\right)\) =1 i=1,2,...,N,σi and θ=1,2,...,M.

If U>0, then we found that Zσθ=−Pσθ; this is impossible because Zσθ>0 and 0 ≤Pσθ≤1. Thus, if U=0, and subtracting (8) from (6), we have,

$${\begin{aligned} &\left.\left(\sigma \lambda_{\theta }^{\sigma -1}-\frac{(1-\lambda_{\theta }^{\sigma })}{T_{\theta }}\right) P_{\sigma \theta }e^{-\left(\frac{Z_{\sigma \theta }}{T_{\theta }}\right) }\right.\\ &\prod\nolimits_{\substack{ i=1 \\ i\neq \sigma }}^{N}{\sum\nolimits}_{j=1}^{M}\left[ \left(1-\lambda_{j}^{i}\right) P_{ij}e^{-\left(\frac{Z_{ij}}{T_{j}}\right) }+6{\sum\nolimits}_{\varpi }\left(1-\lambda_{j+\varpi }^{i}\right)P_{i(j+\varpi)}e^{-\left(\frac{Z_{i(j+\varpi)}}{T_{j+\varpi }}\right) }\right] =0. \end{aligned}} $$

Then, we have,

$$ \left(\sigma \lambda_{\theta }^{\sigma -1}-\frac{\left(1-\lambda_{\theta }^{\sigma }\right)}{T_{\theta }}\right) P_{\sigma \theta }e^{-\left(\frac{ Z_{\sigma \theta }}{T_{\theta }}\right) }=0; $$
(11)

or

$$ \prod\limits_{\substack{ i=1 \\ i\neq \sigma }}^{N}\sum\limits_{j=1}^{M} \left[ \left(1-\lambda_{j}^{i}\right) P_{ij}e^{-\left(\frac{Z_{ij}}{T_{j}} \right) }+6\sum\limits_{\varpi }\left(1-\lambda_{j+\varpi }^{i}\right)P_{i(j+\varpi)}e^{-\left(\frac{Z_{i(j+\varpi)}}{T_{j+\varpi }}\right) }\right] =0. $$
(12)

Since the probability of the first target in the cell j is greater than zero, then \(\left. P_{\sigma (\theta +\varpi)}e^{-\left (\frac {Z_{\sigma (\theta +\varpi)}}{T_{\theta +\varpi }}\right) }>0\right. \). In addition, Tj is a factor due to the search in cell j and the dimensions of it (it is a given value where this value returns to the nature of the searching process). Consequently, we obtain the optimal value of \(\lambda _{j}^{\ast }\) at time step i from (11) by solving the equation: \(i\lambda _{j}^{i-1}- \frac {\left (1-\lambda _{j}^{i}\right)}{T_{j}}=0\), this leads to:

$$ \lambda_{j}^{i}+iT_{j}\lambda_{j}^{i-1}-1=0. $$
(13)

Similarly, by subtracting (9) from (7), we have,

$$\begin{aligned} &{\sum\nolimits}_{\varpi }6\left(\sigma \lambda_{\theta +\varpi }^{\sigma -1}- \frac{\left(1-\lambda_{\theta +\varpi }^{\sigma }\right)}{T_{\theta +\varpi }}\right) P_{\sigma (\theta +\varpi)}e^{-\left(\frac{Z_{\sigma (\theta +\varpi)}}{ T_{\theta +\varpi }}\right) }\\ & \times \prod\nolimits_{\substack{ i=1 \\ i\neq \sigma }}^{N}{\sum\nolimits}_{j=1}^{M}\left[ \left(1-\lambda_{j}^{i}\right) P_{ij}e^{-\left(\frac{Z_{ij}}{T_{j}}\right) }\,+\,6{\sum\nolimits}_{\varpi }\left(1-\lambda_{j+\varpi }^{i}\right)P_{i(j+\varpi)}e^{-\left(\frac{Z_{i(j+\varpi)}}{T_{j+\varpi }}\right) }\right]\! =\!0. \end{aligned} $$

This gives the optimal value of \(\lambda _{j+\varpi }^{\ast }\) at time step i by solving the following equation:

$$ \lambda_{j+\varpi }^{i}+iT_{j+\varpi }\lambda_{j+\varpi }^{i-1}-1=0. $$
(14)

Let \(r_{i}=E\left (X_{i}\right) -K_{p}\sqrt {Var\left (X_{i}\right) },\) then from (10) we get,

$${\sum\nolimits}_{j=1}^{M}\left[ \left(1-\lambda_{j}^{i}\right)Z_{ij}+{\sum\nolimits}_{\varpi }\left(1-\lambda_{j+\varpi }^{i}\right)Z_{i(j+\varpi)}\right] -r_{i}=0, $$

at least one of these boundaries satisfies that,

$$ \left(1-\lambda_{j}^{i}\right)Z_{ij}+{\sum\nolimits}_{\varpi }\left(1-\lambda_{j+\varpi }^{i}\right)Z_{i(j+\varpi)}-r_{i}=0. $$
(15)

Also, from (12), we conclude that at least one of these boundaries satisfies such that,

$$ \left(1-\lambda_{j}^{i}\right) P_{ij}e^{-\left(\frac{Z_{ij}}{T_{j}} \right) }+6\sum\limits_{\varpi }\left(1-\lambda_{j+\varpi }^{i}\right)P_{i(j+\varpi)}e^{-\left(\frac{Z_{i(j+\varpi)}}{T_{j+\varpi }}\right) }=0. $$
(16)

From (15), (16) and by subsisting with \(\lambda _{j}^{\ast }\) and \( \lambda _{j+\varpi }^{\ast }\), we get

$$ {}Z_{ij}\,=\,\ln \left[ \frac{\left(1-\lambda_{j}^{i\ast }\right) P_{ij}}{\left(1-\lambda_{j}^{i\ast }\right) Z_{ij}+\sum_{\varpi }\left[ \left(1-\lambda_{j+\varpi }^{i\ast }\right)\left(Z_{i(j+\varpi)}-6P_{i(j+\varpi)}e^{-\left(\frac{Z_{i(j+\varpi)}}{T_{j+\varpi }}\right) }\right) \right] -r_{i}}\right]^{T_{j}} $$
(17)

If we know the optimal effort \(Z_{i(j+\varpi)}^{\ast }\), then from (17) in (15), we get:

$$ Z_{ij}^{\ast }=P_{ij}e^{-\left(\frac{r_{i}-\sum_{\varpi }\left(1-\lambda_{j+\varpi }^{i\ast }\right)Z_{i(j+\varpi)}^{\ast }}{T_{j}\left(1-\lambda_{j}^{i\ast }\right)}\right) }-\left(\frac{\sum_{\varpi }\left(1-\lambda_{j+\varpi }^{i\ast }\right)Z_{i(j+\varpi)}^{\ast }-r_{i}}{\left(1-\lambda_{j}^{i\ast }\right)}\right) $$
(18)

Also, if we know the optimal effort \(Z_{ij}^{\ast }\), we can get \( Z_{i(j+\varpi)}^{\ast }\) from solving the following equation:

$$ {\begin{aligned} &(1-\lambda_{j}^{i\ast })\left[ P_{ij}e^{-\left(\frac{r_{i}-\sum_{ \varpi }\left(1-\lambda_{j+\varpi }^{i\ast }\right)Z_{i(j+\varpi)}^{\ast }}{ T_{j}\left(1-\lambda_{j}^{i\ast }\right)}\right) }-\left(\frac{\sum_{\varpi }\left(1-\lambda_{j+\varpi }^{i\ast }\right)Z_{i(j+\varpi)}^{\ast }-r_{i}}{\left(1-\lambda_{j}^{i\ast }\right)}\right) \right] \\&\quad+\sum\limits_{\varpi }\left(1-\lambda_{j+\varpi }^{i\ast }\right)Z_{i(j+\varpi)}^{\ast }-r_{i}=0. \end{aligned}} $$
(19)

By knowing the minimum values \(\lambda _{j}^{\ast }\), \(\lambda _{j+\varpi }^{\ast },Z_{ij}^{\ast }\) and \(Z_{i(j+\varpi)}^{\ast }\), we can obtain the minimum value of H(Z;λ). This minimum values will maximize the probability of detecting the targets with minimum cost.

An algorithm

We use the following dynamic programming algorithm in contribution to solve larger instances of our problem to obtain the minimum search effort. The steps of the algorithm can be summarized as follows: Step 1. Insert the total number of time intervals N and the total number of cells M, E(Xi), Var(Xi), Kp, the probability of the initial state of the first target P0, and the one-step transition probability matrix P. Step 2. At time interval i, use P and P0 to generate \(\bar {P} _{ij}=P_{ij}+{\sum \nolimits }_{\varpi }P_{i(j+\varpi)}\) the transition probability matrix of the two targets. Based on some recent information about the expected location of the other target, we can let \( A_{i}={\sum \nolimits }_{\varpi }P_{i(j+\varpi)},\varpi =-h-1,-h,-h+1,-1,1,h-1,h,h+1.\) Thus, one can obtain the value of \(\bar {P} _{ij}.\) Step 3. Calculate the values of λj and λj+ϖ from Eqs. (11) and (12), respectively. Step 4. By the given values of E(xi) and Var(xi) at each time interval i=1,2,...,N, input the values of ri where \(r_{i}=E\left (X_{i}\right) -K_{p}\sqrt {Var\left (X_{i}\right) },\) elsewhere go to step 8. Step 5. From equations (18) and (19), compute the values of Zij,Zi(j+ϖ), elsewhere go to step 8. Step 6. Substitute with the value of λj,λj+ϖ,Zij,Zi(j+ϖ),Pij,Ai in (4) to compute the value of H(Z). Now, put j=j+1, if jM, then return to step 2, else put i=i+1 and test the condition iN if yes then go to step 2 else go to step 7. Step 7. Give the total value of H(Z) and then stop. Step 8. End (stop).

This algorithm works to estimate the minimum value of λj,λj+ϖ,Zij and Zi(j+ϖ) where in step 1 we input the total number of N and M. In addition, we insert the values of E(Xi) and Var(Xi) during each time interval i=1,2,...,N. Based on the values of P0 and P, we calculate the value of Pij as in step 2. By considering the values of \( A_{i}={\sum \nolimits }_{\varpi }P_{i(j+\varpi)},\varpi =-h-1,-h,-h+1,-1,1,h-1,h,h+1,\) then we get the probability of detecting the two targets during the time interval i in the cell j is given by \(\bar {P} _{ij}=P_{ij}+A_{i}.\) At time interval i, the algorithm computes the values of λj and λj+ϖ as in step 3 and the value of ri where \(r_{i}=E\left (x_{i}\right) +K_{p}\sqrt {Var\left (x_{i}\right) }\) as in step4. After that, the algorithm goes tostep 5and computes Zij,Zi(j+ϖ) from (18) and (19) respectively. Now all anonymous values become known, then go to step 6; else, end the process. At the end ofstep 6, compute the value of H(Z). Do all the above steps for all time intervals and all cells whenever the conditions jM,iN are satisfied. Finally, in step 7, give the total value of H(Z) and then end the process.

One Markovian target

In this section, we will consider two cases for one Markovian target as follows.

Applying discount effort case

In the case of one target, the above DNLSP is equivalent to the following nonlinear stochastic programming problem (NLSP(2)),

NLSP(2):

$$\begin{aligned} \underset{Z_{ij},\lambda_{j}}{\min} \qquad & H(Z;\lambda)=\prod\nolimits_{i=1}^{N}{\sum\nolimits}_{j=1}^{M}\left[ \left(1-\lambda_{j}^{i}\right) P_{ij}e^{-\left(Z_{ij}/T_{j}\right)}\right], \\ & \textit{sub. to \ } \mathit{Z}\left(X\right) \mathit{=}\left\{ Z\in R^{NM}\mid g(Z;\lambda)={\sum\nolimits}_{j=1}^{M}\left[ \left(1-\lambda_{j}^{i}\right)Z_{ij}\right] \right. \\ &\quad \quad\left. -E\left(X_{i}\right) -K_{p}\sqrt{ Var\left(X_{i}\right) }\leq 0\right\}, \\ & Z_{ij}\geq 0,0<\lambda_{j}<1, {\sum\nolimits}_{j=1}^{M}P_{j}=1 \textit{\ }\forall \textit{\ }i=1,2,...,N\ \text{and}\ j=1,2,...,M\textit{\ . } \end{aligned} $$

Then, from (6),(8), and (10), we have,

$$ -\frac{\left(1-\lambda_{\theta }^{\sigma }\right)P_{\sigma \theta }}{T_{\theta }}.e^{-\left(\frac{Z_{\sigma \theta }}{T_{\theta }}\right) }\prod\limits_{\substack{ i=1 \\ i\neq \sigma }}^{N}\sum\limits_{j=1}^{M}\left[ \left(1-\lambda_{j}^{i}\right) P_{ij}e^{-\left(\frac{Z_{ij}}{T_{j}}\right) } \right] +U\left(1-\lambda_{\theta }^{\sigma }\right) =0, $$
(20)
$$ -\sigma \lambda_{\theta }^{(\sigma -1)}P_{\sigma \theta }.e^{-\left(\frac{ Z_{\sigma \theta }}{T_{\theta }}\right) }\prod\limits_{\substack{ i=1 \\ i\neq \sigma }}^{N}\sum\limits_{j=1}^{M}\left[ \left(1-\lambda_{j}^{i}\right) P_{ij}e^{-\left(\frac{Z_{ij}}{T_{j}}\right) })\right] -U\sigma \lambda_{\theta }^{\sigma -1}Z_{\sigma \theta }=0, $$
(21)
$$ U\left\{ \sum\limits_{j=1}^{M}\left[ \left(1-\lambda_{j}^{i}\right)Z_{ij}\right] -r_{i}\right\} =0, $$
(22)

If U>0, then we found that Zσθ=−TθPσθ; this is impossible because Zσθ,Tθ>0 and 0 ≤Pσθ≤1. Thus, if U=0, and subtracting (21) from (20), we have,

$$ \lambda_{j}^{i}+iT_{j}\lambda_{j}^{i-1}-1=0. $$
(23)

which is the same result as in (13) (this gives \(\lambda _{j}^{\ast }\)). In addition, to obtain \(Z_{ij}^{\ast }\), we found that at least one of the boundaries for (21),(22) and (23) (where U=0) equal to 0 as follows:

$$ \left(1-\lambda_{j}^{i}\right) P_{ij}e^{-\left(\frac{Z_{ij}}{T_{j}} \right) }=0, $$
(24)
$$ \left[ \left(1-\lambda_{j}^{i}\right)Z_{ij}\right] -r_{i}=0. $$
(25)

Then, we have \(\left (1-\lambda _{j}^{i}\right) P_{ij}e^{-\left (\frac {Z_{ij} }{T_{j}}\right) }=\left [ \left (1-\lambda _{j}^{i}\right)Z_{ij}\right ] -r_{i}\) which gives,

$$ Z_{ij}=\ln \left[ \frac{\left(1-\lambda_{j}^{i\ast }\right) P_{ij}}{\left(1-\lambda_{j}^{i\ast }\right) Z_{ij}-r_{i}}\right]^{T_{j}} $$
(26)

Also, (26) can be obtained from (17) after substituting with

$${\sum\nolimits}_{\varpi }\left[ \left(1-\lambda_{j+\varpi }^{i\ast }\right)\left(Z_{i(j+\varpi)}-6P_{i(j+\varpi)}e^{-\left(\frac{Z_{i(j+\varpi)}}{ T_{j+\varpi }}\right) }\right) \right] =0. $$

Thus, one can get:

$$ Z_{ij}^{\ast }=P_{ij}e^{-\left(\frac{r_{i}}{T_{j}\left(1-\lambda_{j}^{i\ast }\right)} \right) }+\frac{r_{i}}{\left(1-\lambda_{j}^{i\ast }\right)} $$
(27)

The optimal value of undetecting probability function is given by:

$$ H(Z^{\ast };\lambda^{\ast })=\prod\limits_{i=1}^{N}\sum\limits_{j=1}^{M} \left[ \left(1-\lambda_{j}^{i\ast }\right) P_{ij}\text{ exp}\left[ -\frac{ P_{ij}e^{-\left(\frac{r_{i}}{T_{j}\left(1-\lambda_{j}^{i\ast }\right)}\right) }+\frac{ r_{i}}{\left(1-\lambda_{j}^{i\ast }\right)}}{T_{j}}\right] \right] $$
(28)

Without applying discount effort case

Here, we do not use the discount effort function or we put λj=0 in the above NLSP(2), then we need to minimize the searching effort Zij only. This makes the above NLSP(2) will take the form:

NLSP(3):

$$\begin{aligned} &\underset{Z_{ij}}{\min} \qquad H(Z)=\prod\nolimits_{i=1}^{N}{\sum\nolimits}_{j=1}^{M} \left[ P_{ij}e^{-\left(Z_{ij}/T_{j}\right)}\right],\\ &\textit{sub. to \ }\mathit{Z}\left(X\right) \mathit{=}\left\{ Z\in R^{NM}\mid g(Z)={\sum\nolimits}_{j=1}^{M}\left[ Z_{ij}\right] -E\left(X_{i}\right) -K_{p}\sqrt{Var\left(X_{i}\right) }\leq 0\right\},\\ & Z_{ij}\geq 0,{\sum\nolimits}_{j=1}^{M}P_{j}=1 \textit{\ }\forall \textit{\ } i=1,2,...,N\ \text{and}\ j=1,2,...,M \textit{\ . } \end{aligned} $$

By applying the Kuhn-Tucker conditions, we have,

$$ -\frac{P_{\sigma \theta }}{T_{\theta }}.e^{-\left(\frac{Z_{\sigma \theta }}{ T_{\theta }}\right) }\prod\limits_{\substack{ i=1 \\ i\neq \sigma }} ^{N}\sum\limits_{j=1}^{M}\left[ P_{ij}e^{-\left(\frac{Z_{ij}}{T_{j}} \right) }\right] +U=0, $$
(29)
$$ U\left\{ \sum\limits_{j=1}^{M}Z_{ij}-r_{i}\right\} =0, $$
(30)

Leads to,

$$ Z_{ij}=\ln \left[ \frac{P_{ij}}{Z_{ij}-r_{i}}\right]^{T_{j}} $$
(31)

Using (27), we have,

$$ Z_{ij}^{\ast }=P_{ij}e^{-\left(\frac{r_{i}}{T_{j}}\right) }+r_{i} $$
(32)

The optimal value of undetecting probability function is given by:

$$ H(Z^{\ast })=\prod\limits_{i=1}^{N}\sum\limits_{j=1}^{M}\left[ P_{ij}\exp\left[ -\frac{P_{ij}e^{-\left(\frac{r_{i}}{T_{j}}\right) }+r_{i}}{T_{j}} \right] \right] $$
(33)

Randomly located target

Let the probability of the target in cell j, j=1,2,...,M, be πj. After the cell j has been searched, the searcher may either continue to search the same cell or switch without any delay to another cell. The searching process in each cell is conducted independently of previous searches and takes one unit of time. Thus, if the target has been stated in the cell j with probability ξj, where 0<ξj<1, Song and Teneketizs [3] showed that the probability of detecting the target in the ith time interval is Pij=πjξj(1−ξj)i−1,i=1,2,...,N; j=1,2,...,M. Consequently, in the case of applying the discount effort function case (applying discount effort case) as in NLSP(2), we get the equivalent optimization problem,

NLSP(4):

$$\begin{aligned} & \underset{Z_{ij},\lambda_{j}}{\min }\qquad H(Z;\lambda)=\prod\nolimits_{i=1}^{N}{\sum\nolimits}_{j=1}^{M}\left[ \left(1-\lambda_{j}^{i}\right) \left(\pi_{j}\xi_{j}(1-\xi_{j})^{i-1}\right)e^{-\left(Z_{ij}/T_{j}\right)})\right], \\ & \textit{sub. to \ }\mathit{Z}\left(X\right) \mathit{=}\left\{ Z\in R^{NM}\mid g(Z;\lambda)={\sum\nolimits}_{j=1}^{M}\left[ \left(1-\lambda_{j}^{i}\right)Z_{ij}\right] \right. \\ & \quad \quad \left. -E\left(X_{i}\right) -K_{p}\sqrt{ Var\left(X_{i}\right) }\leq 0\right\},\\ & Z_{ij}\geq 0,0<\lambda_{j}<1, {\sum\nolimits}_{j=1}^{M}\pi_{j}\xi_{j}\left(1-\xi_{j}\right)^{i-1}=1 \forall i=1,2,...,N\ \text{and} \ j=1,2,...,M. \end{aligned} $$

As in applying discount effort case, we get

$$ \lambda_{j}^{i}+iT_{j}\lambda_{j}^{i-1}-1=0, $$
(34)
$$ Z_{ij}^{\ast }=\pi_{j}\xi_{j}\left(1-\xi_{j}\right)^{i-1}e^{-\left(\frac{r_{i}}{ T_{j}\left(1-\lambda_{j}^{i\ast }\right)}\right) }+\frac{r_{i}}{\left(1-\lambda_{j}^{i\ast }\right)}, $$
(35)

and the optimal value H(Z;λ) is given by:

$$ {\begin{aligned} &H(Z^{\ast };\lambda^{\ast })\\&=\prod\limits_{i=1}^{N}\sum\limits_{j=1}^{M} \left[ \left(1-\lambda_{j}^{i\ast }\right) \left(\pi_{j}\xi_{j}(1-\xi_{j})^{i-1}\right)\text{ exp}\left[ -\frac{\left(\pi_{j}\xi_{j}(1-\xi_{j})^{i-1}\right)e^{-\left(\frac{r_{i}}{T_{j}\left(1-\lambda_{j}^{i\ast }\right)}\right) }+ \frac{r_{i}}{\left(1-\lambda_{j}^{i\ast }\right)}}{T_{j}}\right] \right]. \end{aligned}} $$
(36)

In addition, if we do not apply the discount effort in NLSP(4), then we get the following optimization problem,

NLSP(5):

$$\begin{aligned} &\underset{Z_{ij}}{\min }\qquad H(Z)=\prod\nolimits_{i=1}^{N}{\sum\nolimits}_{j=1}^{M} \left[ \left(\pi_{j}\xi_{j}\left(1-\xi_{j}\right)^{i-1}\right)e^{-\left(Z_{ij}/T_{j}\right)}\right], \\ & \textit{sub. to \ }\mathit{Z}\left(X\right) \mathit{=}\left\{ Z\in R^{NM}\mid g(Z)={\sum\nolimits}_{j=1}^{M}Z_{ij}-E\left(X_{i}\right) -K_{p} \sqrt{Var\left(X_{i}\right) }\leq 0\right\}, \\ & Z_{ij}\geq 0, {\sum\nolimits}_{j=1}^{M} \pi_{j}\xi_{j}\left(1-\xi_{j}\right)^{i-1}=1\textit{\ }\forall \textit{\ } i=1,2,...,N \ \text{and} \ j=1,2,...,M\textit{\ .} \\ \textit{\ } \end{aligned} $$

and the optimal value of \(\lambda _{j}^{\ast }\) at time step i is given from solving the equation (13) or (23) or (34). Also, the optimal values of \(Z_{ij}^{\ast }\) and H(Z) are given by:

$$ Z_{ij}^{\ast }=\left(\pi_{j}\xi_{j}\left(1-\xi_{j}\right)^{i-1}\right)e^{-\left(\frac{r_{i}}{ T_{j}}\right) }+r_{i}, $$
(37)
$$ H(Z^{\ast })=\prod\limits_{i=1}^{N}\sum\limits_{j=1}^{M}\left[ \left(\pi_{j}\xi_{j}\left(1-\xi_{j}\right)^{i-1}\right)\exp\left[ -\frac{\left(\pi_{j}\xi_{j}\left(1-\xi_{j}\right)^{i-1}\right)e^{-\left(\frac{r_{i}}{T_{j}}\right) }+r_{i}}{T_{j}} \right] \right]. $$
(38)

Application

We will consider the above dynamic programming algorithm in the above cases and compare between them to show the effectiveness of our model. Now, consider a Markovian target moves on two states with a transition matrix

$$Q=\left[\begin{array}{cc} 0.2 & 0.8 \\ 0.4 & 0.6 \end{array}\right], $$

with initial probabilities: \( P_{01}=\frac {3}{5},P_{02}=\frac {2}{5}\) and Tj=j, j=1,2, i=1,2,3. The probabilities Pi1 and Pi2 are \(\frac {2}{3}-\left \{(0.4)^{i-1}\right \}/15\) and \(\frac {1}{3}+\left \{(0.4)^{i-1}\right \}/15\) for i=1,2,3, respectively (see Bhat [40]). In addition, let Xi has a normal distribution with mean E(xi)=0.82 and variance Var(xi)=0.04. We assume that the standard normal random variable Kp takes the values {3,4,5} and λj={0.4,0.8} to obtain the optimal values of Zij, i=1,2,3, j=1,2 from (27) and H(Z;λ) from (28), see Table 1.

Table 1 The values of Zij,i=1,2,3,j=1,2 and H(Z;λ) for arbitrary values of ri

When we do not use the discount effort reward function and using the above assumption in this application, we get the optimal values of Zij, i=1,2,3, j=1,2 (from (32)) and H(Z) (from (33)) as in Table 2.

Table 2 The values of Zij,i=1,2,3,j=1,2 and H(Z) for arbitrary values of ri without using λ

From the numerical calculations, we found that the value of H(Z;λ) (see Table 1) is very small than the value of H(Z) (see Table 2). This shows the effectiveness of our model. That happens although the values of Zij,i=1,2,3,j=1,2 in Table 2 are greater than the values of them in Table 1. Really this is true but when we use the discount effort reward function, the optimal values of Zij are calculated from \(\left (1-\lambda _{j}^{i\ast }\right)Z_{ij}^{\ast }\) as in Table 3, where λ1=0.4,λ2=0.8.

Table 3 The optimal values of Zij,i=1,2,3,j=1,2 when we use the discount effort reward function for a Markovian target

This shows that the values of \(Z_{ij}^{\ast },i=1,2,3,j=1,2\) in the case of using the discount effort reward function are smaller than the value of them in the other case.

On the other hand, if the probability of the target in the cell j, j=1,2 be π1=0.2, π2=0.8, respectively, and if we consider the target has been stated in the cell j with probability ξ1=0.4, ξ1=0.6, and when we use the discount effort reward function, the optimal values of Zij, i=1,2,3, j=1,2 are calculated from (35) and H(Z;λ) from (36), see Table 4.

Table 4 The values of Zij,i=1,2,3,j=1,2 and H(Z;λ) for a randomly located target when we use the discount effort reward function

In without applying the discount effort reward function case, we get the optimal values of Zij, i=1,2,3, j=1,2 (from (37)) and H(Z) (from (38)) as in Table 5.

Table 5 The values of Zij,i=1,2,3,j=1,2 and H(Z) for a randomly located target when we do not use the discount effort reward function

Also, we see that the value of H(Z;λ) in Table 4 is very small than the value of H(Z) in Table 5. From Table 4, the optimal values of Zij,i=1,2,3,j=1,2 are greater than the values of them in Table 5. Thus, the optimal values of Zij are calculated from \( \left (1-\lambda _{j}^{i\ast }\right)Z_{ij}^{\ast }\) as in Table 6, where λ1=0.4,λ2=0.8.

Table 6 The optimal values of Zij,i=1,2,3,j=1,2 when we use the discount effort reward function for a randomly located target

As in Table 6, the values of \(Z_{ij}^{\ast },i=1,2,3,j=1,2\) in the case of using the discount effort reward function are smaller than the value of them in the other case.

Conclusion and future research

A new method has been presented to give the maximum discounted effort reward and the minimum possible cost for detecting two related targets (i. e., the targets which are related together in the movement). This method is different from the method which has been presented in El-Hadidy [5]. We minimize the values of the search effort Zij, the tuning parameter λj, and the probability of undetected Pij, i=1,2,...,N and j=1,2,...,Mat the same time. We present some special cases of one Markovian and hidden target. The experimental results are obtained from detecting two targets; one of them moves with a Markov process, and the other is randomly located. Also, compare these results in two cases, considering and ignoring the discount effort reward.

In future works, we will investigate and analyze the stability of NLSP(1), NLSP(2), NLSP(3), NLSP(4), and NLSP(5) by characterizing the set of feasible discounted effort reward parameters. Also, we can study the related dual problem of these problems. Also, this model is more suitable for using the multiple searchers case by considering the combinations of movement of multiple targets.

Availability of data and materials

Not applicable.

References

  1. 1

    Hong, S., Cho, S., Park, M.: A pseudo-polynomial heuristic for path-constrained discrete-time Markovian-target search. Eur. J. Oper. Res. 193, 351–364 (2009).

    MathSciNet  MATH  Google Scholar 

  2. 2

    Hong, S., Cho, S., Park, M., Lee, M.: Optimal search-relocation trade-off in Markovian-target searching. Comput. Oper. Res. 36, 2097–2104 (2009).

    MATH  Google Scholar 

  3. 3

    Song N, Teneketizs, D: Discrete search with multiple sensors. Math. Meth. Oper. Res. 60, 1–13 (2004).

    MathSciNet  Google Scholar 

  4. 4

    Mohamed, A., Kassem, M., El-Hadidy, M.: M-states search problem for a lost target with multiple sensors. Int. J. Math. Oper. Res. 10(1), 104–135 (2017).

    MathSciNet  Google Scholar 

  5. 5

    El-Hadidy, M.: ‘On maximum discounted effort reward search problem’. Asia Pac. J. Oper. Res. 33(3), 30 (2016). 1650019.

    MathSciNet  MATH  Google Scholar 

  6. 6

    El-Hadidy, M.: Fuzzy optimal search plan for N-dimensional RandomlyMoving Target. Int. J. Comput. Methods. 13(6), 38 (2016). 1650038.

    MathSciNet  MATH  Google Scholar 

  7. 7

    Kassem, M., El-Hadidy, M.: Opimal multiplicative Bayesian search for a lost target. Appl. Math. Comput. 247, 795–802 (2014).

    MathSciNet  MATH  Google Scholar 

  8. 8

    El-Hadidy, M.: Optimal searching for a helix target motion. Sci. China Math. 58(4), 749–762 (2015).

    MathSciNet  MATH  Google Scholar 

  9. 9

    Mohamed, A., El-Hadidy, M.: Optimal multiplicative generalized linear search plan for a discrete random walker. J. Optim. 2013(Article ID 706176), 13 (2013). doi:10.1155/2013/706176.

    MATH  Google Scholar 

  10. 10

    Mohamed, A., Kassem M., El-Hadidy, M.: Multiplicative linear search for a brownian target motion. Appl. Math. Model. 35(9), 4127–4139 (2011).

    MathSciNet  MATH  Google Scholar 

  11. 11

    El-Hadidy M.: Searching for a d-dimensional Brownian target with multiple sensors. Int. J. Math. Oper. Res. 9(3), 279–301 (2016).

    MathSciNet  Google Scholar 

  12. 12

    El-Hadidy, M., Kassem, M.: On minimum expected search time for a multiplicative random search problem. Int. J. Oper. Res. 29(2), 219–247 (2017).

    MathSciNet  Google Scholar 

  13. 13

    El-Hadidy, M., Abou-Gabal H.: Optimal searching for a randomly located target in a bounded known region. Int. J. Comput. Sci. Math. 6(4), 392–403 (2015).

    MathSciNet  Google Scholar 

  14. 14

    El-Hadidy, M.: Optimal spiral search plan for a randomly located target in the plane. Int. J. Oper. Res. 22(4), 454–465 (2015).

    MathSciNet  MATH  Google Scholar 

  15. 15

    El-Hadidy, M., Abou-Gabal H.: ‘Coordinated search for a random walk target motion’. Fluctuation Noise Lett. 17(1), 11 (2018). 1850002.

    Google Scholar 

  16. 16

    El-Hadidy M.: Generalised linear search plan for a D-dimensional random walk target. Int. J. Math. Oper. Res. 15(2), 211–241 (2019).

    MathSciNet  Google Scholar 

  17. 17

    El-Hadidy, M.: Study on the three players’ linear rendezvous search problem. Int. J. Oper. Res. 33(3), 297–314 (2018).

    MathSciNet  Google Scholar 

  18. 18

    Beltagy, M., El-Hadidy, M.: Parabolic spiral search plan for a randomly located target in the plane. ISRN Math. Anal. 2013(Article ID 151598), 8 (2013). doi:10.1155/2013/151598.

    MathSciNet  MATH  Google Scholar 

  19. 19

    El-Hadidy, M., Alzulaibani, A.: Cooperative search model for finding a Brownian target on the real line. J. Taibah Univ. Sci. 13(1), 177–183 (2019).

    Google Scholar 

  20. 20

    El-Hadidy, M.: On the existence of a finite linear search plan with random distances and velocities for a one-dimensional Brownian target. Int. J. Oper. Res. 37(2), 245–258 (2020).

    MathSciNet  Google Scholar 

  21. 21

    El-Hadidy, M.: Study on the three players linear rendezvous search problem. Int. J. Oper. Res. 33(3), 297–314 (2018).

    MathSciNet  Google Scholar 

  22. 22

    El-Hadidy, M.: Existence of finite parbolic spiral search plan for a Brownian target. Int. J. Oper. Res. 31(3), 368–383 (2018).

    MathSciNet  Google Scholar 

  23. 23

    El-Hadidy, M., Teamah, A., El-Bagoury, A.: 3-Dimensional coordinated search technique for a randomly located target. Int. J. Comput. Sci. Math. 9(3), 258–272 (2018).

    MathSciNet  Google Scholar 

  24. 24

    El-Hadidy, M., El-Bagoury, A.: Optimal search strategy for a three-dimensional randomly located target. Int. J. Oper. Res. 29(1), 115–126 (2017).

    MathSciNet  Google Scholar 

  25. 25

    Mohamed, A., Abou Gabal, H., El-Hadidy, M.: Random search in a bounded area. Int. J. Math. Oper. Res. 10(2), 137–149 (2017).

    MathSciNet  Google Scholar 

  26. 26

    Mohamed, A., El-Hadidy M.: Existence of a periodic search strategy for a parabolic spiral target motion in the plane. Afrika Matematika J. 24(2), 145–160 (2013).

    MathSciNet  MATH  Google Scholar 

  27. 27

    Mohamed, A., Abou-Gabal H, El-Hadidy, M.: Coordinated search for a randomly located target on the plane. Eur. J. Pur. Appl. Math. 2(1), 97–111 (2009).

  28. 28

    Mohamed, A., Fergany, H., El-Hadidy, M.: On the coordinated search problem on the plane. J. Sch. Bus. Adm. Istanbul Univ. 41(1), 80–102 (2012).

    Google Scholar 

  29. 29

    El-Hadidy, M., Alzulaibani, A.: Existence of a finite multiplicative search plan with random distances and velocities to find a d-dimensional Brownian target. J. Taibah Univ. Sci. 13(1), 1035–1043 (2019).

    Google Scholar 

  30. 30

    El-Hadidy, M.: Existence of cooperative search technique to find a Brownian target. J. Egypt. Math. Soc. 28(1), 1–12 (2020).

    MathSciNet  MATH  Google Scholar 

  31. 31

    El-Hadidy, M., Alfreedi, A.: Minimizing the expected search time of finding the hidden object by maximizing the discount effort reward search. J. Taibah Univ. Sci. 14(1), 479–487 (2020).

    Google Scholar 

  32. 32

    El-Hadidy, M., Alzulaibani, A.: A mathematical model for preventing HIV virus from proliferating inside CD4 T brownian cell using Gaussian jump nanorobot. Int. J. Biomath. 12(07), 24 (2019). 1950076.

    MathSciNet  MATH  Google Scholar 

  33. 33

    El-Hadidy, M., Abou-Gabal, H.: Searching for the random walking microorganism cells. Int. J. Biomath. 12(6), 12 (2019). 1950064.

    MathSciNet  MATH  Google Scholar 

  34. 34

    El-Hadidy, M.: Studying the finiteness of the first meeting time between Lévy flight jump and Brownian particles in the fluid reactive anomalous transport. Mod. Phys. Lett. B. 33(22), 8 (2019). 1950256.

    MathSciNet  Google Scholar 

  35. 35

    El-Hadidy, M., Alzulaibani, A.: Study on the finiteness of the first meeting time between N-dimensional Gaussian jump and Brownian diffusion particles in the fluid. Int. J. Mod. Phys. B. 33(28), 22 (2019). 1950334.

    Google Scholar 

  36. 36

    El-Hadidy, M., Alfreedi, A.: On optimal coordinated search technique to find a randomly located target. Stat. Optim. Inf. Comput. 7(4), 854–863 (2019).

    MathSciNet  Google Scholar 

  37. 37

    Luo, J., Wang, S., Zhang, E.: Signal detection based on a decreasing exponential function in alpha-stable distributed noise. KSII Trans. Internet Inf. Syst. 12(1), 269–286 (2018).

    Google Scholar 

  38. 38

    Blum, A., Chawla, S., Karger, D., Lane, T., Meyerson, A., Minkoff, M.: Approximation algorithms for orienteering and discounted-reward TSP. In: Proc 44th Annual IEEE Symp of Computer Science, pp. 46–55. IEEE, Cambridge, MA (2003).

    Google Scholar 

  39. 39

    Mangasarian, O.: Nonlinear Programming. MC Grow Hill, Inc New York, London (1969).

    MATH  Google Scholar 

  40. 40

    Bhat, U.: Elements of Applied Stochastic Processes. John Wiley Sons, New York (1971).

    Google Scholar 

Download references

Acknowledgements

The author gratefully acknowledges the anonymous referees for their insightful and constructive comments and suggestions.

Funding

Not applicable.

Author information

Affiliations

Authors

Contributions

The author has made each part of this paper. He read and approved the final manuscript.

Corresponding author

Correspondence to Mohamed Abd Allah El-Hadidy.

Ethics declarations

Competing interests

The author declares that he has no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Abd Allah El-Hadidy, M. The searching algorithm for detecting a Markovian target based on maximizing the discounted effort reward search. J Egypt Math Soc 28, 37 (2020). https://doi.org/10.1186/s42787-020-00097-1

Download citation

Keywords

  • Search theory
  • Probability theory
  • Discounted effort reward search
  • Markovian targets

AMS Subject Classification

  • 37A50
  • 60K30
  • 90B40