Human-robot interaction (HRI) control spans two coupled layers: a physical layer, in which a robot exchanges forces, positions, and motion with a human partner in direct or teleoperated contact, and a cognitive/social layer, in which the robot must interpret intent, adapt to preferences, and coordinate actions with a human whose behaviour is only partially predictable. Classical solutions to the physical layer, impedance and admittance control^[1], render the robot’s response to contact forces through a fixed or scheduled mechanical relationship. These schemes are provably stable, but they are built on a model of the environment (and implicitly, of the human) that is rarely accurate: human arm stiffness, reaction time, and intent all vary across users and across a single task^[2].

Deep reinforcement learning (DRL) offers an alternative: rather than hand-specifying an interaction law, the robot learns a policy directly from interaction data, allowing the impedance/admittance behaviour itself to become adaptive and human-specific^[3,4]. This shift from designed to learned interaction control is the central subject of this survey.

The shift is not free, and the price is paid in a currency that HRI can least afford. A learned interaction law is a black box placed in mechanical contact with a person, so the two questions that classical control answered by construction, will it remain stable and will it remain safe, must now be answered statistically, from finite data, about a policy that continues to change after deployment. Three consequences organise this survey. First, sample efficiency stops being an engineering convenience and becomes an ethical constraint, because every sample is a physical interaction with a human body; this is what makes demonstration data, offline pre-training, and simulation transfer central rather than optional. Second, the environment is non-stationary in a specific and awkward way: the human adapts to the robot while the robot adapts to the human, so the stationarity assumption underlying the Markov decision process is violated by the very success of the learning process. Third, guarantees must be stated in a form that survives learning, which is why uncertainty quantification and set-invariance methods have moved from the periphery of this literature to its centre in the last three years.

The same pattern, a perception-decision-action loop whose reliability must be certified because a person is on the receiving end, is visible well beyond manipulation. Reviews of embodied AI in clinical settings organise the field along exactly this axis and reach a similar conclusion about the translation bottleneck, namely that the barrier is evaluation and standardisation rather than raw capability^[5]. We draw on that parallel where it is informative, while keeping the scope of this survey on physical and cognitive HRI control.

Four developments justify a new survey rather than a simple update of our 2021 book Human-Robot Interaction Control Using Reinforcement Learning^[6]: (i) safe and stability-certified RL has matured from Lyapunov-style heuristics into formal H₂/uniform ultimate boundedness (UUB) guarantees^[7]; (ii) conformal-prediction-based uncertainty quantification has entered control design^[8]; (iii) large language and vision-language-action models now condition manipulation policies on natural-language instruction^[9,10]; and (iv) residual and hybrid reinforcement learning (RL), learning a correction on top of a model-based controller rather than a policy from scratch, has become the dominant paradigm for sample-efficient, safety-preserving deployment^[4,11].

Table 1 organises the literature underlying this survey into nine paradigms that trace the field’s progression from classical, fixed-law HRI control to certified and language-conditioned DRL. This classification also fixes the vocabulary used throughout the sections that follow.

The remainder of this survey is organised as follows. Section 2 bridges classical HRI control and DRL. Section 3 proposes a four-part classification of DRL approaches for HRI. Section 4 discusses emerging directions, Section 5 lists open challenges, and Section 6 concludes.

Admittance and impedance control regulate the dynamic relationship between interaction force F_h (applied by the human) and robot motion. A standard admittance model relates force to a commanded position correction x_d through a virtual mass-damper-spring system,

(1) $$ M \ddot{x}_d + D \dot{x}_d + K x_d = F_h , $$

where M, D, and K are the designer-chosen virtual inertia, damping, and stiffness^[1]. Stability of Equation (1) is straightforward when F_h is bounded and slowly varying, but M, D, K are typically fixed offline; they do not adapt to a specific human’s stiffness, fatigue, or intent. A large body of work addresses this by scheduling the gains online: variable admittance schemes modulate D from measured interaction power or estimated user intent^[12,13], and variable impedance learning treats the stiffness profile itself as the object to be adapted^[3,14]. In all of these the interaction law still relies on a parametric structure fixed at design time.

Reinforcement learning removes that structural assumption. The HRI problem is cast as a Markov decision process ($$ \mathcal{S} $$, $$ \mathcal{A} $$, P, r, γ), where the state s_t ∈ $$ \mathcal{S} $$ includes robot and (where observable) human kinematic/force data, the action a_t ∈ $$ \mathcal{A} $$ is a motion or force command, and the policy π_θ(a_t | s_t) is trained to maximize expected discounted return,

(2) $$ \pi^* = \arg\max_{\pi} \; \mathbb{E}_{\pi}\!\left[ \sum_{t=0}^{\infty} \gamma^t\, r(s_t, a_t) \right], $$

with the reward r(s_t, a_t) typically shaped to penalize excessive interaction force, task error, and unsafe configurations. Three properties of the human partner make Equation (2) attractive over the fixed model in Equation (1):

• Unknown human dynamics. The mapping from robot behaviour to human response is rarely known in closed form; RL treats it as part of the environment transition P rather than requiring an explicit model. Evidence from human motor control supports this treatment: people co-adapt force and impedance rather than holding a fixed mechanical law^[15], so the mapping being learned is genuinely dynamic rather than merely unknown.

• Non-stationarity. Human stiffness, attention, and intent drift within and across sessions. Model-free policies can be updated online as this drift occurs, whereas fixed-gain admittance laws [Equation (1)] cannot.

• Contact uncertainty. Under unknown or time-varying environment constraints, residual strategies that learn a correction on top of a nominal force controller^[4] and demonstration-guided exploration^[16,17] have been shown to recover stable behaviour faster than learning from scratch.

Figure 1 illustrates this shift schematically: the classical loop (dashed) computes x_d directly from F_h via Equation (1); the DRL loop (solid) instead routes (s_t, F_h) through a learned policy π_θ, whose parameters are updated from the interaction reward, closing the loop at the learning level rather than only at the control level.

We organise the DRL-for-HRI literature into four sub-categories, summarised in Table 2 and illustrated in Figure 2, that differ in what is being learned (a control law, a policy from data, a joint human-robot strategy, or a certified policy) rather than in application domain. The four categories build on the general DRL-for-robotics methodology established by policy-search and end-to-end visuomotor formulations^[31,32], which they specialise to the case of a human in the loop.

The four categories are not disjoint, and their overlaps are significant for design. Demonstration learning (Section 3.2) and shared autonomy (Section 3.3) differ mainly in when the human’s influence enters: in the former the human supplies an offline dataset or a preference label that shapes r before deployment, whereas in the latter the human is an agent whose action a_t^h appears inside the value function at run time. The boundary blurs as soon as demonstrations are collected online, at which point a demonstration-learning system is a shared-autonomy system with a particular arbitration rule. Safe RL (Section 3.4) is better read as orthogonal to the other three than as a peer of them: it is a constraint layer that can be applied to any of them. Concretely, demonstration data is what makes a safety filter tractable, because it supplies the calibration set that a conformal bound [Equation (7)] needs^[28], while shared autonomy is what makes a safety filter necessary, because a non-stationary human partner is precisely the disturbance that a fixed-margin barrier cannot absorb^[29]. Reading the four categories as a 3+1 structure, three answers to “where does human information enter” plus one certification layer, explains why the most deployment-ready systems in Table 3 combine categories rather than choosing between them.

Three observations follow from Table 3.

Deployment readiness runs inverse to conceptual ambition. The category that assumes least about the human, admittance shaping, is the one actually running on industrial hardware, while the category that models the human most richly, shared autonomy, has produced almost no deployed systems. This is not a transient state of maturity. Modelling the human as a rational agent buys expressiveness at the cost of an assumption that cannot be validated on the small cohorts these studies use, so the added realism is difficult to falsify in practice. In selecting a method for a given application, the assumptions listed in the second column are therefore the appropriate starting point.

The assumptions fail together, not independently. Each row’s assumption is violated by the same underlying fact, that the human adapts. Non-stationarity invalidates the fixed gains of 3.1, shifts the demonstration distribution of 3.2, breaks the stationary-policy premise of 3.3, and destroys the exchangeability that the conformal certificates of 3.4 require. Combining categories therefore does not average the risk away, because the failures are correlated: a system that pairs offline demonstrations with a conformal safety filter has two components that become miscalibrated at the same moment and for the same reason. Reported robustness gains from combining methods should be read with this in mind.

The data requirements in column four are not commensurable, which makes published comparisons unreliable. A demonstration costs human time, a preference label costs human attention, and a simulated rollout costs neither. Sample counts are nonetheless reported in a single currency, so a method requiring 10⁴ simulated episodes is routinely described as less data-hungry than one requiring 10² human comparisons, when for a rehabilitation patient the ordering is reversed. Until the field reports human-cost separately from environment-interaction count, claims of improved sample efficiency in HRI are not comparable across papers. We regard this as a more tractable near-term fix than any algorithmic change, and as a concrete instance of the benchmarking challenge discussed in Section 5.

Three directions extend the classification of Section 3 beyond what was addressed in^[6] and are summarised in Table 4.

Large language model (LLM)/Vision-language-action (VLA)-conditioned policies. Vision-language-action models such as RT-2^[9] condition a manipulation policy directly on a natural-language instruction and an image observation, π_θ(a_t | s_t, ℓ), where ℓ is the instruction embedding. Combined with task-and-motion planning^[10], this allows an HRI system to accept unstructured verbal requests rather than a single fixed reward function, a qualitative departure from every method in Section 3, all of which assume a task-specific r fixed at training time. The trajectory here is not unique to robotics: medical vision-language analysis has passed through the same three stages, from task-specific models, through adapter- and prompt-tuned variants, to generalist foundation models, and the reported gains there are in data efficiency and cross-domain generalisation rather than in peak single-task accuracy^[38]. That parallel is worth taking seriously as a prediction: if manipulation follows the same curve, the argument for VLA-conditioned policies in HRI will rest on few-shot adaptation to a new user, not on outperforming a well-tuned task-specific controller on the task it was tuned for.

Digital-twin-based sim-to-real transfer. Real-world DRL training in physical contact with a human is costly and risky; a digital twin of the human-robot-environment system allows π_θ to be pre-trained under Equation (2) in simulation, with the safety mechanisms of Section 3.4 used to certify the policy before, rather than during, real-world deployment. The transfer techniques this relies on, domain randomisation, system identification, and progressive adaptation, are surveyed in^[37]; surveys of real-world DRL deployment in robotics consistently flag the residual sim-to-real gap in human-facing settings as an open bottleneck^[33], motivating this direction as a priority rather than a solved problem.

Despite the progress organised in Tables 1 and 2, five challenges remain largely open across the DRL-for-HRI literature we surveyed. Table 5 summarises them; each is discussed below.

Each challenge below is stated as a specific unresolved question rather than a topic, together with what would count as an answer. Several were surfaced by the comparative analysis of Section 3 and are, to our knowledge, not posed in this form elsewhere.

C1. Is per-user conservatism the right regulariser for offline HRI? Offline RL penalises actions outside the behaviour distribution^[39,40], but Section 4.1 argued that the shift that matters in HRI is between the logged human and the deployed human, not between behaviour and learned policy. The concrete question: can a conservatism penalty be defined over an estimated user-parameter posterior, for instance over arm stiffness and reaction latency, rather than over actions? An answer would be an offline method whose pessimism relaxes as the deployed user’s estimated parameters approach the logged cohort, and it could be evaluated today on existing real-robot offline benchmarks^[41] extended with multi-user data.

C2. Can conformal coverage be made valid in closed loop? Section 3.4 showed that conformal sets now compose with set-invariance guarantees in HRI^[26-29] but not with Lyapunov/UUB stability, because the robot’s action changes the human’s next motion and so destroys exchangeability. The specific open problem is to bound the disturbance term in $$ \dot{V} $$(e_t) using a coverage statement that remains valid under this feedback. Adaptive conformal prediction^[27] shows one route, abandoning fixed calibration; a second, untried route is to calibrate on interventional rather than observational data, that is, on trajectories in which the robot’s action was deliberately randomised. The answer would be a UUB bound whose ε in Equation (6) carries an explicit, finite-sample confidence level.

C3. Do the four categories fail independently? The synthesis following Table 3 argued they do not: human adaptation invalidates the assumptions of all four simultaneously. If correct, this predicts that a system pairing offline demonstrations with a conformal safety filter will show correlated degradation, both components decalibrating within the same session. This is directly testable, and to our knowledge untested: instrument a combined system, log calibration error and policy regret separately, and measure their correlation across a session in which the user adapts. A near-zero correlation would falsify the claim and justify treating the categories as independent risk reducers; a high correlation would mean that published robustness gains from combining methods are overstated.

C4. How many users, of what diversity, are enough? Most reported results are single-user or small-cohort^[12,22]. The unresolved question is not qualitative but quantitative: what is the sample complexity of user generalisation? A concrete first step is a scaling study holding the algorithm fixed and varying cohort size and diversity, reporting held-out-user performance as a curve rather than a single number. Without such a curve there is no principled basis for the cohort sizes currently used, which appear to be set by recruitment convenience.

C5. Report human cost separately from interaction count. As argued after Table 3, demonstrations, preference labels and simulated rollouts are aggregated into a single sample count even though they draw on incommensurable resources, so a method needing 10⁴ simulated episodes is described as more efficient than one needing 10² human comparisons. This item requires no new algorithm. We propose that HRI-DRL papers report a three-element cost vector: environment interactions, human minutes, and number of distinct participants. Unlike a full benchmark suite, which requires community coordination^[33,35], this convention can be adopted by an individual paper without prior agreement across the field.

This survey organised DRL for HRI into four paradigms, physical HRI via admittance shaping, learning from demonstration and feedback, multi-agent and shared-autonomy control, and safe, stability-certified RL, and traced their evolution from the fixed-law control reviewed in our 2021 book^[6] to the certified, language-conditioned, and residual formulations emerging today. Relative to recent DRL-robotics surveys in which HRI appears as one application area among many^[33], and to surveys addressing a specific HRI setting or a specific communication channel rather than control-theoretic guarantees^[34,35], we focused specifically on what a robot can be guaranteed to do while interacting with a human. Section 3.4 argues that the residual weakness is not the absence of distribution-free uncertainty quantification in HRI, which now exists^[26-29], but its confinement to set-invariance rather than stability guarantees.

Three directions are, in our view, the most consequential for closing that gap: reconciling conformal-prediction coverage^[8], which is now routinely composed with set-invariance guarantees in HRI^[26-29], with Lyapunov/UUB stability results^[7] into a single certification framework, which requires a coverage argument that survives the loss of exchangeability in closed loop; using digital twins to move safety validation from real-world trials into simulation before deployment; and adopting residual RL [Equation (8)] as a default architecture, since it inherits the guarantees of a nominal controller while retaining the adaptability that motivated moving beyond classical admittance control [Equation (1)] in the first place. We hope the classification and open challenges identified here [Tables 2 and 5] provide a concrete agenda for both the RL and HRI communities.