Private, Edge and Federated Inference
Computation should move only after a system names what would cross each boundary and proves that the chosen placement clears privacy, quality, latency, energy and availability gates.
TLDR
- Running a model near the data can reduce transfers, but physical proximity is not a privacy guarantee. Privacy depends on an explicit adversary, disclosure and retention model.
- A split is defensible only after measuring each possible boundary: local compute, transmitted bytes, network delay, device energy, task quality and the information recoverable from intermediate representations.
- Use hard gates for prohibited disclosures and minimum service levels, then compare the remaining device, edge and cloud plans on a Pareto frontier rather than one invented “privacy score”.
- Encrypted transport, trusted hardware, cryptographic inference, differential privacy and federated collaboration protect different assets. None repairs a compromised endpoint or careless telemetry by itself.
- The architecture decision changes from “local or cloud?” to “which information may leave, under which protection, for what gain, and what happens when the network or device fails?”
Two phones photograph the same identity document. Phone A runs its classifier locally, then uploads an embedding, a crash snapshot and a diagnostic trace to three separate services. Phone B performs deterministic redaction on the device, sends only approved fields to a controlled remote service, disables content logging and deletes transient buffers after a verified result. Which phone provides the more private inference path?
The words local and remote do not answer. Phone A keeps the main neural computation on the handset yet creates several disclosures. Phone B crosses a network boundary but may expose less information to fewer principals. Even that comparison remains incomplete until the adversary is named. A malicious application on the phone, a curious service operator, a network observer, a model thief and an authorised analyst abusing retained logs see different attack surfaces.
Compute placement is therefore an information-boundary problem before it is an infrastructure preference. Device, edge and cloud describe where instructions execute. Privacy describes what a specified observer can learn or cause. Federated learning describes how training updates are combined. Split inference describes one forward pass divided across tiers. Emerging uses of federated inference describe collaborative serving, but the term is not yet governed by one settled mechanism.
The practical consequence is demanding but useful. First record every representation that could move or persist. Next reject plans that violate disclosure, quality, latency, energy, availability or recovery requirements. Only then optimise the feasible plans. A neat topology diagram cannot substitute for that evidence.
Topology is not privacy
Edge computing arose from a real systems pressure. Sensors, phones, vehicles and industrial equipment produce data far from central data centres. Sending every byte to a distant service consumes bandwidth, adds variable delay and makes the application depend on a wide-area path. Edge-computing accounts therefore emphasise proximity, responsiveness, bandwidth and local context.[1][2] These are valuable properties. They do not make every nearby processor trustworthy, every remote processor unsafe or every local result private.
Consider a smart speaker that performs wake-word detection locally. The design avoids continuously streaming ambient audio before a wake event. That is a meaningful reduction in raw-data movement. Yet the device could still retain audio after the event, transmit a transcript, expose speaker embeddings, upload failure samples or reveal household routines through timing metadata. The local detector changes the disclosure surface; it does not close it.
This is why privacy should be framed through effects on people and data processing, rather than through one favoured infrastructure label. The NIST Privacy Framework similarly treats privacy risk as potential problems arising from systems that process data across their life cycle.[20] Its framework is voluntary and broad; it does not choose an inference topology. It does reinforce the right starting point: identify the processing, actors and consequences before selecting controls.
Four different placements
All-local inference keeps the forward pass on the originating device. It can work without a network and can prevent an operator from seeing query content, provided the application, operating system, model package, logs and update channel honour that boundary. It also moves model weights into a physically exposed environment and inherits the device's memory, thermal and energy limits.
Remote inference sends an input or derived request to an edge or cloud service. It can use larger models, pooled accelerators and centrally controlled releases. The price is a disclosure to the serving boundary, network dependence and a larger operational estate. Encryption in transit protects the bytes from ordinary path observers, not from the endpoint that decrypts them for computation.
Split or collaborative inference executes an early segment locally and later layers remotely. Neurosurgeon formulated the split as a latency and energy optimisation over mobile and cloud resources, while Edgent combined device-edge partitioning with early exits.[4][5] A split can reduce local computation or raw-input movement. It also creates a new transmitted object: the intermediate activation.
Federated or multi-owner inference lets several devices, models or organisations contribute to one query-time result. The participants may exchange activations, encrypted shares, votes, logits, proofs or carefully limited requests. This is an inference-time protocol. It should not be confused with federated learning, where clients compute training updates from local data and an aggregator combines those updates into a model.
Take a system that sends an image embedding to a service, retains it for seven days and exposes it to the same operator. In World A the service rack is called “edge” and sits five kilometres away. In World B it is called “cloud” and sits five hundred kilometres away. Keep encryption, identity, retention, operator access, model and output unchanged. The network delay may change. The operator's access to the embedding does not.
Now reverse the intervention. Keep the rack in the same building, but change the protocol so that it receives a cryptographic share that is insufficient on its own, logs no content and returns only a bounded result. The privacy properties can change while the location remains fixed. Geography can affect exposure, but it cannot stand in for an exposure model.
The disclosure ledger
A useful placement decision starts with six possible disclosures. The first is the raw input, including metadata that may be more identifying than the payload. The second is an intermediate representation, such as a feature tensor, embedding, token sequence or partial state. The third is the output, including probabilities, explanations and repeated answers. The fourth is the model, whose weights, architecture or prompts may be valuable intellectual property. The fifth is operating residue: traces, caches, crash dumps, queues and backups. The sixth is any update sent for adaptation or learning.
For each object, record who can observe it, whether observation is plaintext or protected, how long it persists, what other data can be joined to it and what effect the observer may cause. A tensor that looks anonymous in isolation may become identifying when paired with a stable account, timestamp or device fingerprint. A diagnostic sample retained “temporarily” may outlive the model release that created it.
Every layer is a possible border
A neural network is usually drawn as one model, but a placement engine sees a sequence of transformations. After each stage lies a possible cut. The cut determines how much work remains local, which representation crosses the network and what the remote side can observe. It may also determine which weights are exposed on a customer-owned device and whether the system can continue during an outage.
The tempting rule is to push the cut “deeper” until the representation becomes small and abstract. That rule fails twice. First, activation volume is not monotonic. Early convolutions can expand channels faster than spatial downsampling shrinks them, so an intermediate tensor may exceed the input. Second, abstraction is not erasure. Mahendran and Vedaldi reconstructed photographically recognisable content from several studied CNN representations.[7] Later work has attacked collaborative inference directly by reconstructing unseen inputs from intermediate features.[10]
A split index is therefore neither a byte count nor a privacy level. Each candidate cut needs three measurements: the cost of producing the representation, the cost of moving it and an attack-specific test of what it reveals.
The cost of a split
Suppose a model has layers 1 through L, and layers up to k run on the device. A first latency estimate is:
Cdevice is measured device compute time for the local segment. Sk is the transmitted representation in kilobytes. B is effective bandwidth in megabits per second, so the factor eight converts bytes to bits under the stated decimal units. RTT is round-trip delay, Qremote is queueing and Cremote is remote compute. Protocol setup, serialisation, radio wake-up and retries should be added when material.
Changing k moves several terms at once. A later cut usually adds local compute and device energy. It may reduce remote compute, but it may increase or decrease transmitted bytes. It can change quality if the boundary quantises, compresses, noises or exits early. The same plan may perform differently under a warm Wi-Fi connection, a congested mobile path, a thermally throttled phone and a low-battery policy.
Energy deserves its own measurement rather than inference from latency. Local acceleration consumes processor and memory energy; transmission consumes radio energy and may keep a modem in a high-power state. MLPerf Tiny made accuracy, latency and energy distinct benchmark dimensions for constrained inference.[6] The relevant denominator is also task-specific: millijoules per accepted inference, not merely instantaneous power or operations per second.
Model design is another candidate before partitioning. MobileNets made width and input resolution explicit levers for trading computation against task performance on mobile vision workloads.[3] Quantisation, pruning, distillation and smaller task models can similarly change the feasible set. A split optimiser that compares only one oversized network across locations may “discover” a network dependency that disappears when the local baseline is properly sized.
Build two networks with the same number of layers and cut both after layer three. Network A preserves a near-identity residual path and emits a high-resolution feature map. Network B passes through a narrow, task-specific bottleneck trained against a reconstruction and sensitive-attribute attacker. The layer number, device and server remain unchanged. Only the representation changes.
If an adversary reconstructs faces from A but not B, the privacy difference belongs to representation and attack resistance, not depth. If B also loses the feature needed for the authorised task, its lower leakage is not a free improvement. Privacy Partitioning and DeepObfuscator exemplify this utility-adversary trade by explicitly training or evaluating intermediate representations against private-attribute and reconstruction objectives.[8][9]
Measure a boundary as an attack surface
“The tensor is unintelligible to a person” is not a test. A serious boundary assessment gives the attacker the access expected in operation: observed activations, auxiliary public data, knowledge of architecture, perhaps query access and perhaps a related training distribution. It then measures reconstruction, attribute inference, linkage or membership success against a baseline. The result is scoped to that attack, dataset and model version.
Protection can fail after a model change even when the cut index is stable. Fine-tuning may encode a sensitive attribute more strongly. Quantisation may remove useful task information before it removes private information. A compression layer can reduce bytes without reducing recoverability. The boundary test must therefore travel with the model release, rather than surviving as a one-time architecture assertion.
Worked example 1: a minimal partitioning lab
The accompanying laboratory models a six-stage evidence classifier. Raw input begins on the device. The full model reaches synthetic task quality 0.944 but consumes 163 milliseconds and 81 millijoules when run entirely locally. A private bottleneck after stage four emits 96 KB with attack-risk score 0.22. Sending that representation to a nearby edge tier yields 141.6 milliseconds and 64.2 millijoules under the stated network profile.
The policy first applies hard gates: quality at least 0.940, latency no more than 155 milliseconds, device energy no more than 70 millijoules, no raw crossing and activation risk no more than 0.30. The full local plan fails latency and energy. Earlier splits fail disclosure or latency because they move larger, more revealing activations. The split after the private bottleneck is the only Pareto-feasible full-model plan in this synthetic case.
The lab also contains a serious negative control. A compact model reaches quality 0.933, 62 milliseconds and 35 millijoules entirely on the device. Under the strict quality gate it is correctly rejected. Lower the required quality to 0.930, without changing any system measurement, and the compact local baseline wins. When a simple local model is good enough, network partitioning has not earned its complexity.
inference_partition_lab.py. Synthetic stage, link and risk values; deterministic assertions verify both cases.The core enumerator is deliberately small. It accepts only monotonic device-to-edge-to-cloud placements, calculates transfer cost at every boundary and applies policy gates before selection. The abridged excerpt below exposes the gate order; the linked file is the runnable artefact:
def evaluate(model, placement, policy):
latency = sum(stage.latency_ms[tier]
for stage, tier in zip(model.stages, placement))
energy = sum(stage.device_energy_mj
for stage, tier in zip(model.stages, placement)
if tier == "device")
raw_crosses, risks = False, []
for boundary in boundaries(placement):
link = LINKS[boundary.source, boundary.target]
kb, risk, raw = payload_before(model, boundary.index)
latency += link.rtt_ms + link.queue_ms + kb * 8 / link.bandwidth_mbps
energy += device_radio_energy(boundary, kb)
risks.append(risk)
raw_crosses |= raw
failed = []
if model.quality < policy.min_quality: failed.append("quality")
if latency > policy.max_latency_ms: failed.append("latency")
if energy > policy.max_device_energy_mj: failed.append("energy")
if raw_crosses: failed.append("raw-crossing")
if max(risks, default=0) > policy.max_activation_risk:
failed.append("activation-risk")
return Result(feasible=not failed, failed_gates=failed, ...)
Inspect the implementation in this paper. It uses only the Python standard library, prints both policy outcomes, emits JSON on request and asserts that the positive case and negative control select the expected plans.
Assumptions and limits of the executable lab
The values are synthetic. The “activation risk” number stands for a bounded attack evaluation supplied by a real assurance process; the script does not pretend to derive privacy from tensor size. It models one user-privacy threat and does not score model theft, compromised endpoints, side channels, physical attacks, malicious collaborators or output extraction.
A production evaluator should replay measured stage traces from target hardware, network distributions rather than one bandwidth value, thermal and battery states, cold starts, retries, concurrent load, model-version-specific leakage tests and recovery behaviour. It should preserve the vector of results instead of hiding incompatible harms inside one weighted score.
flowchart LR A["Topology is not privacy"] --> B["Every layer is a possible border"] B --> C["Privacy mechanisms protect<br/>different things"] C --> D["Apply hard gates before<br/>optimisation"] D -. "re-test" .-> A classDef first fill:#0a1424,stroke:#087A98,stroke-width:2px,color:#e6f3ff classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff class A first class B,C,D step
Privacy mechanisms protect different things
Once a placement exposes an object to another principal, the next question is what that principal may learn or change. The answer cannot be “we use privacy technology”. Each mechanism assumes an adversary and provides a bounded protection. Combining mechanisms can strengthen a design, but only when their assumptions compose.
Transport encryption protects the path
TLS or an equivalent protected channel prevents ordinary network observers from reading or modifying traffic between authenticated endpoints. It is essential for remote inference. It does not conceal plaintext from the client process before encryption, from the serving process after decryption, from content logs or from an authorised operator with service access. Calling a conventional HTTPS inference call “private inference” confuses transport confidentiality with endpoint confidentiality.
Metadata also remains. Destination, timing, volume and recurrence can reveal routines or sensitive events even when payloads are encrypted. Padding, batching, relays and local suppression can reduce some metadata leakage, but each adds latency, cost or complexity. The design must state whether metadata is in scope.
Trusted execution changes the server boundary
A trusted execution environment aims to isolate code and data from the surrounding host, often with remote attestation so a client can verify the measured workload before releasing secrets. This can reduce trust in the operating system or cloud administrator. It transfers trust to hardware, firmware, attestation, enclave code, key provisioning and side-channel resistance. It also constrains memory and observability.
Slalom demonstrated a useful form of partitioning inside this mechanism: keep sensitive control inside a trusted environment while outsourcing expensive linear operations to an untrusted accelerator, with privacy and verification measures around the delegated work.[13] The paper establishes a specific protocol and performance result on studied networks. It does not make all accelerator delegation confidential, or all trusted hardware immune to leakage.
Cryptographic inference hides values through computation
Homomorphic encryption permits selected operations on ciphertexts. Secure multi-party computation lets parties jointly compute a function while limiting what each party learns beyond the permitted output. CryptoNets showed that neural inference over encrypted inputs was possible on MNIST; Gazelle combined homomorphic linear algebra with two-party techniques for non-linear layers.[11][12] These works are important demonstrations, not evidence that any current model can be wrapped with negligible cost.
The architecture, activation functions, numeric precision, communication rounds, batching and adversary model all matter. Some protocols protect client inputs but reveal model architecture. Others aim to protect both input and weights. Semi-honest and malicious security are different claims. A secure protocol can still release an over-informative output or be embedded in a service that logs plaintext before encryption.
Differential privacy limits an observable distribution
Differential privacy is not a synonym for adding noise. It bounds how much a specified output distribution changes when one protected unit is added or removed, under a stated privacy budget and composition rule. The protected unit could be a person, record, event, device or query. These choices change the meaning of the guarantee.
In training, differential privacy can constrain what model parameters reveal about individual examples. At inference, noise may be applied to activations, outputs or aggregate statistics, but the guarantee must be derived for that release mechanism. Repeated queries consume privacy or create averaging attacks unless the protocol accounts for them. Noise that defeats one attribute classifier does not automatically establish differential privacy.
Output and operating controls remain necessary
Even a perfect confidential-computing path can expose information through its permitted output. Membership inference asks whether a record was in the training set by exploiting differences in model behaviour; Shokri and colleagues demonstrated the risk in black-box settings.[17] Prediction interfaces can also support model extraction. Controls therefore include output minimisation, calibration, rate limits, query anomaly detection, purpose binding and refusal of requests that are too revealing.
Logging is a separate release. A trace may contain prompts, tensors, identifiers, confidence scores or decrypted errors. Privacy-preserving computation followed by plaintext debug retention merely moves the breach point. Telemetry should default to typed operational events, with content capture treated as an exceptional, consented and time-bounded workflow.
| Mechanism | Primary protection | Does not by itself solve | Evidence required |
|---|---|---|---|
| Protected transport | Confidentiality and integrity between authenticated endpoints. | Endpoint access, content logging, over-informative outputs or compromised clients. | Protocol configuration, endpoint identity, key lifecycle and traffic tests. |
| Trusted execution | Isolation from specified host software and operators, with attestation where used. | Vulnerable enclave code, side channels, hardware assumptions, input or output misuse. | Attestation policy, TCB inventory, patch state, side-channel assessment and fail-closed tests. |
| HE or MPC | Values hidden from defined computational parties under a cryptographic model. | Leakage through outputs, client compromise, wrong circuit, protocol denial of service. | Security model, implementation review, numeric equivalence, latency and communication profile. |
| Differential privacy | Bounded influence of a protected unit on a specified release. | Arbitrary secrecy, compromised endpoints, unlimited unaccounted composition. | Neighbour relation, epsilon and delta, accountant, clipping, utility and repeated-release tests. |
| Output controls | Reduced extraction, membership or inference surface at the interface. | Internal access, training-data defects, side channels or stolen weights. | Query threat model, rate policy, output ablations, attack simulation and incident telemetry. |
Federated learning is not federated inference
Federated learning changes where training data is processed. In the canonical FedAvg pattern, clients compute local updates and a coordinator aggregates them to improve shared model parameters.[14] Secure aggregation can prevent the server from seeing each participant's individual update while still recovering an aggregate.[15] These mechanisms do not make the resulting query-time service local, and federated training does not automatically prevent update leakage. Deep Leakage from Gradients showed that shared gradients can reveal training examples under studied conditions.[16]
Federated inference instead concerns one or more predictions. Recent work uses the label in at least two ways. A SIGCOMM 2025 poster describes heterogeneous edge devices collaboratively executing a partitioned model, with capability-aware cuts and differential-privacy noise on activations.[18] A 2026 preprint frames independently trained, privately owned models collaborating at inference time without sharing their data or parameters, and explicitly notes that a unified abstraction is still lacking.[19]
The term is emerging, so an architecture must define it operationally. State the participants, object exchanged, aggregation rule, privacy mechanism, failure semantics, incentive assumptions and authority to return a result. Otherwise “federated” imports the positive connotation of data locality without specifying what the inference protocol actually reveals.
| Dimension | Federated learning | Federated inference |
|---|---|---|
| When | During optimisation or adaptation. | During one prediction or decision-support request. |
| Typical object exchanged | Gradients, parameter deltas, sufficient statistics or secure shares. | Activations, logits, votes, encrypted shares, proofs, sub-queries or partial outputs. |
| Persistent result | Updated model parameters or training state. | Prediction, evidence object or bounded query-time state. |
| Main systems constraint | Client sampling, non-IID data, update communication, convergence and poisoning. | End-to-end latency, availability, correlated errors, input privacy and output authority. |
| Privacy question | What do updates or final weights reveal about local training data? | What does each collaborator learn about this query, other models and the returned result? |
The explicit failure boundary
This paper's placement method stops helping when the trusted endpoint itself is under adversarial control. A rooted phone can read raw input before a local model transforms it. A malicious display process can capture the result after decryption. Physical probes may recover weights or keys. Side channels can violate an otherwise correct cryptographic or enclave model. A tampered update can replace the protected bottleneck with an exfiltration path.
The method also fails when leakage cannot be tested against a meaningful adversary, or when the task has no stable quality and consequence threshold. In those cases a numeric partition optimiser creates false precision. The appropriate decision may be to avoid remote inference, avoid retaining the input, narrow the task, require human handling or refuse automation. Placement optimises within a permitted system boundary; it cannot make an impermissible purpose acceptable.
Worked example 2: card-dispute evidence intake
Consider a synthetic mobile journey in which a customer photographs a receipt and selects a disputed transaction. The goal is to prepare an evidence packet, not to decide the dispute. Images may contain names, addresses, partial card details and unrelated purchases. Devices vary widely. Network quality may collapse halfway through an upload.
On the phone, deterministic code checks file type, orientation and size. Optical extraction and a small model propose text regions. A local redaction stage removes fields that the case policy does not permit to leave the device. A private bottleneck converts approved regions into a compact representation and attaches a manifest naming the source image, transformations, model version and redaction confidence. Low-confidence redaction causes a rescan or human review rather than remote escalation of the raw image.
A regional edge service receives the representation under a scoped workload identity. A larger model proposes merchant, amount, date and evidence-category fields. Deterministic validators check formats and reconcile extracted values with the selected transaction. The service returns typed fields plus evidence references, not a free-form conclusion. The central case system records the proposal, policy version and effect receipt. A person remains accountable for any consequential disposition.
This arrangement is not automatically superior to an all-local model. It is justified only if the larger remote stage earns enough quality on difficult documents and the protected boundary survives attack testing. The compact local baseline remains available during outage and for documents it handles confidently. The remote path is an escalation, not the default definition of intelligence.
Apply hard gates before optimisation
An architecture team often asks for a single score to rank device, edge and cloud plans. That is attractive and dangerous. A weighted sum can trade a prohibited disclosure against a few milliseconds of latency, or offset unacceptable quality with a lower cloud bill. Some requirements are not preferences. They are admission conditions.
Let a placement plan p have task quality q, tail latency l, device energy e, availability a and a disclosure vector d. Define the feasible set before choosing a winner:
qmin is the minimum quality supported by task evidence. l95 is a tail-latency requirement rather than a favourable average. emax reflects the device and usage pattern. amin includes the network, remote service and local fallback. The disclosure terms include zero-tolerance events, such as raw biometric material crossing an unapproved boundary, and measured attack thresholds for representations that are allowed to move.
Only plans in F should enter cost optimisation. Within that set, preserve a Pareto frontier when no plan is better on every relevant dimension. A lower-energy plan may have higher latency. A more accurate plan may require a protected remote tier. The decision maker can then see the exchange instead of inheriting a hidden weight chosen by the optimiser author.
Availability changes the privacy decision
Remote compute introduces a failure mode that all-local inference may avoid. A responsible design specifies what happens when the edge is unreachable, slow or unable to attest. Retrying may duplicate cost or disclosure. Falling back to raw cloud upload may violate the reason for the local boundary. Returning an unverified result may convert an availability incident into a correctness incident.
Possible responses include a smaller local model, a delayed queue with encrypted state, a request for manual completion or a clear statement that the function is temporarily unavailable. The fallback must meet its own quality and disclosure conditions. A “graceful degradation” that silently widens data access is not graceful.
Device heterogeneity matters as much as network heterogeneity. A cut that works on the newest phone may exhaust memory on an older one. Thermal throttling can turn a passing latency profile into a failure after several requests. Measure by supported device class and workload distribution, then route only among plans qualified for the current context.
The inference placement record
A concise decision record keeps the topology tied to its evidence. It should be versioned with the model, application, privacy tests and release manifest. The following questions are deliberately ordered. Starting with a preferred vendor or tier reverses the logic.
| Field | Question to answer | Release evidence |
|---|---|---|
| Purpose and authority | What may the system infer, recommend or cause, and who remains accountable? | Task statement, outcome contract, decision rights and prohibited uses. |
| Principals and assets | Who can observe raw input, activations, output, model, telemetry and updates? | Data-flow inventory, identities, retention, access and deletion tests. |
| Candidate cuts | Which sequential or collaborative placements are technically possible? | Stage graph, payload schemas, model fragments and network links. |
| Service profile | What quality, p50 and p95 latency, energy, memory and availability does each plan achieve? | Target-device traces, network distributions, load tests and confidence intervals. |
| Disclosure tests | What can the stated adversary reconstruct, infer, link, extract or retain? | Attack code, baselines, thresholds, datasets, model version and residual risk. |
| Protection contract | Which mechanism protects each crossing, under which assumptions? | Protocol, cryptographic or attestation configuration, keys, TCB and verification. |
| Recovery | What happens on outage, timeout, duplicate, stale release, failed attestation or suspected leakage? | Fallback behaviour, checkpoint, idempotency, rollback and incident exercise. |
| Decision and expiry | Why did this plan beat the negative control, and what change forces re-evaluation? | Feasible frontier, chosen trade-off, owner, review trigger and withdrawal path. |
Requalification triggers
A placement decision expires when any assumption that supports it changes. Triggers include a new model or fine-tune, a revised tokenizer or encoder, a different quantisation mode, new supported devices, altered retention, another telemetry provider, a changed edge region, a new attacker capability, a privacy-budget change or a shift in task consequence. A network optimisation can also be a privacy change when it moves the cut or changes the object transmitted.
Requalification should be proportional rather than ceremonial. A remote accelerator swap may require performance and attestation evidence but no new task-quality study if numerical equivalence is demonstrated. A new private bottleneck requires fresh utility and attack tests even when every API remains stable. A change that invalidates a hard gate should withdraw the route automatically, not wait for a periodic committee.
What would weaken the argument?
The thesis would weaken if physical placement reliably determined privacy across relevant adversaries without inspecting representations, outputs or operating residue. It would also weaken if intermediate leakage fell monotonically with depth across architectures and tasks, making the cut index a dependable proxy. Existing inversion and obfuscation results give reasons not to assume either claim, but they do not prove that every activation is sensitive.
A stronger practical challenge is simpler: perhaps a well-sized local model meets every material requirement for most requests. In that case the negative control wins and the edge path should disappear or become a narrow exception. Conversely, cryptographic or trusted execution may make remote computation feasible without a learned bottleneck. The paper does not privilege one mechanism. It privileges measured disclosure and service evidence over placement folklore.
The method also leaves open how to price harms that cannot be made commensurate. A privacy event, a missed urgent detection and a drained battery do not share a natural unit. Hard gates handle unacceptable regions, and a Pareto frontier exposes the remaining conflict. Accountable human judgement is still required to choose among feasible plans.
stateDiagram-v2 [*] --> Claim state "Four different placements" as Claim state "The disclosure ledger" as Test state "The cost of a split" as Revision Claim --> Test: expose mechanism Test --> Decision: evidence survives Test --> Revision: rival survives Revision --> Claim: narrow or rebuild Decision --> [*]
Compact glossary
- Edge inference
- Inference executed on compute near the data source or user, usually to reduce wide-area latency, bandwidth or dependency. Nearness alone says nothing conclusive about trust.
- Split inference
- One model forward pass divided across two or more compute tiers, with an intermediate representation transmitted at each cut.
- Federated learning
- Training or adaptation in which participants process local data and contribute updates or statistics to a collaborative optimisation process.
- Federated inference
- An emerging label for query-time collaboration across devices or independently owned models. Its exact protocol must be defined rather than inferred from the name.
- Disclosure vector
- A record of which raw inputs, activations, outputs, weights, telemetry and updates each principal can observe, retain or join.
- Private bottleneck
- A task-specific representation designed and tested to preserve authorised utility while limiting a stated reconstruction, attribute or linkage attack.
- Hard gate
- A condition a plan must satisfy before optimisation, such as a prohibited raw-data flow, minimum quality or maximum tail latency.
- Pareto frontier
- The feasible plans for which no other plan is at least as good on every retained dimension and strictly better on one.
sequenceDiagram participant Q as Question participant M as Mechanism participant E as Evidence participant D as Decision Q->>M: Topology is not privacy M->>E: Four different placements E-->>M: compare rival explanations E->>D: Move the border only after naming the disclosure D-->>Q: revise scope or act
Move the border only after naming the disclosure
Private inference is sometimes presented as a race to fit a larger model on a smaller device. Edge inference is presented as a latency shortcut. Federated inference is presented as collaboration without centralisation. Each description catches part of the design space and hides the same decisive question: what information becomes available to which principal at each step?
A defensible answer begins with the objects, not the tiers. Raw inputs, activations, outputs, model fragments, telemetry and updates follow different paths. Each path needs an adversary, retention rule, protection mechanism and failure response. Candidate model cuts then face measured quality, latency, energy, availability and leakage tests on the hardware and network that will actually carry them.
The architecture decision changes accordingly. Do not ask whether the model should be local, at the edge or in the cloud as though one noun could settle privacy. Establish the prohibited flows and minimum service first. Put a compact local baseline beside every elaborate partition. Reject any plan that violates a gate. Preserve the Pareto frontier among those that remain, and record why one trade was accepted.
The executable result captures the discipline. A strict quality requirement can justify sending a protected bottleneck to the edge. The moment a compact local model meets the required quality, it becomes the better answer. Computation moves only when the evidence earns the boundary crossing.
References
- Primary survey and systems vision. Shi, W., Cao, J., Zhang, Q., Li, Y. and Xu, L. Edge Computing: Vision and Challenges. IEEE Internet of Things Journal, 2016.
- Primary perspective. Satyanarayanan, M. The Emergence of Edge Computing. Computer, 2017.
- Primary model paper. Howard, A. G. et al. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. 2017.
- Primary systems paper. Kang, Y. et al. Neurosurgeon: Collaborative Intelligence Between the Cloud and Mobile Edge. ASPLOS, 2017.
- Primary systems paper. Li, E., Zhou, Z. and Chen, X. Edge Intelligence: On-Demand Deep Learning Model Co-Inference with Device-Edge Synergy. 2018.
- Primary benchmark paper. Banbury, C. et al. MLPerf Tiny Benchmark. 2021.
- Primary representation study. Mahendran, A. and Vedaldi, A. Understanding Deep Image Representations by Inverting Them. CVPR, 2015.
- Primary privacy-partitioning paper. Chi, J. et al. Privacy Partitioning: Protecting User Data During the Deep Learning Inference Phase. 2018.
- Primary defence paper. Li, A. et al. DeepObfuscator: Obfuscating Intermediate Representations with Privacy-Preserving Adversarial Learning on Smartphones. 2019.
- Primary attack paper. Yin, Y. et al. Ginver: Generative Model Inversion Attacks Against Collaborative Inference. The Web Conference, 2023.
- Seminal primary paper. Gilad-Bachrach, R. et al. CryptoNets: Applying Neural Networks to Encrypted Data with High Throughput and Accuracy. ICML, 2016.
- Primary secure-inference systems paper. Juvekar, C., Vaikuntanathan, V. and Chandrakasan, A. GAZELLE: A Low Latency Framework for Secure Neural Network Inference. USENIX Security, 2018.
- Primary trusted-execution paper. Tramèr, F. and Boneh, D. Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware. ICLR, 2019.
- Seminal federated-learning paper. McMahan, H. B. et al. Communication-Efficient Learning of Deep Networks from Decentralized Data. AISTATS, 2017.
- Primary cryptographic protocol paper. Bonawitz, K. et al. Practical Secure Aggregation for Privacy-Preserving Machine Learning. CCS, 2017.
- Primary attack paper. Zhu, L., Liu, Z. and Han, S. Deep Leakage from Gradients. NeurIPS, 2019.
- Primary attack paper. Shokri, R., Stronati, M., Song, C. and Shmatikov, V. Membership Inference Attacks Against Machine Learning Models. IEEE Symposium on Security and Privacy, 2017.
- Recent primary poster paper. Fan, B., Su, X., Tarkoma, S. and Hui, P. Federated Inference: Towards Collaborative and Privacy-Preserving Inference over Edge Devices. ACM SIGCOMM Posters and Demos, 2025.
- Recent primary preprint and competing formulation. Seo, J., Catak, F. O., Rong, C. and Jang, J. Federated Inference: Toward Privacy-Preserving Collaborative and Incentivized Model Serving. 2026.
- Official risk framework. National Institute of Standards and Technology. NIST Privacy Framework.