Skip to content

Python: visit function parameter and return annotations in new CFG #21943

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
yoff wants to merge 1 commit into
base: yoff/python-cfg-modelling-exceptions
Choose a base branch
from
+72 −168
Draft

Python: visit function parameter and return annotations in new CFG #21943

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions python/ql/lib/change-notes/2026-06-04-cfg-parameter-annotations.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
---
category: minorAnalysis
---
* The new (shared-CFG-based) Python control flow graph now visits parameter and return type annotations as CFG nodes for function definitions, matching the legacy CFG. This restores annotation-based type tracking through framework models such as FastAPI's `Depends()`, Pydantic request models, Starlette `WebSocket` handlers, and any other models that flow a class reference through `Parameter.getAnnotation()` to identify instances of the annotated class.
63 changes: 59 additions & 4 deletions python/ql/lib/semmle/python/controlflow/internal/AstNodeImpl.qll
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1441,10 +1441,19 @@ module Ast implements AstSig<Py::Location> {

/**
* A function definition expression (visits positional and keyword
* defaults, but NOT PEP 695 type parameters — those bind in an
* annotation scope that nests the function body, so they belong to
* the inner scope's CFG, not the enclosing scope's; the legacy CFG
* also omitted them).
* defaults followed by parameter and return type annotations, but NOT
* PEP 695 type parameters — those bind in an annotation scope that
* nests the function body, so they belong to the inner scope's CFG,
* not the enclosing scope's; the legacy CFG also omitted them).
*
* Evaluation order follows CPython: defaults are pushed first, then
* keyword-only defaults, then annotations (the `__annotations__` dict
* is built last, before `MAKE_FUNCTION`). Annotations are emitted as
* CFG nodes so that flows from a class reference into a parameter's
* type annotation are visible to dataflow (e.g. so that framework
* models like FastAPI's `Depends()` can use a parameter's type hint
* to track that the parameter receives an instance of the annotated
* class — see `LocalSources::annotatedInstance`).
*/
additional class FunctionDefExpr extends Expr {
private Py::FunctionExpr funcExpr;
Expand All @@ -1468,15 +1477,61 @@ module Ast implements AstSig<Py::Location> {
rank[n + 1](Py::Expr d, int i | d = funcExpr.getArgs().getKwDefault(i) | d order by i)
}

/**
* Gets the `n`th annotation expression, in CPython evaluation
* order: positional parameter annotations (by argument position),
* `*args` annotation, keyword-only parameter annotations (by
* argument position), `**kwargs` annotation, then the return
* annotation. Each annotation appears at most once.
*/
Expr getAnnotation(int n) {
result.asExpr() =
rank[n + 1](Py::Expr a, int subOrder, int subIndex |
functionAnnotation(funcExpr, a, subOrder, subIndex)
|
a order by subOrder, subIndex
)
}

int getNumberOfDefaults() { result = count(funcExpr.getArgs().getADefault()) }

int getNumberOfKwDefaults() { result = count(funcExpr.getArgs().getAKwDefault()) }

int getNumberOfAnnotations() {
result = count(Py::Expr a | functionAnnotation(funcExpr, a, _, _))
}

override AstNode getChild(int index) {
result = this.getDefault(index)
or
result = this.getKwDefault(index - this.getNumberOfDefaults())
or
result = this.getAnnotation(index - this.getNumberOfDefaults() - this.getNumberOfKwDefaults())
}
}

/**
* Holds if `a` is an annotation of `funcExpr` in slot
* `(subOrder, subIndex)`. Slots are CPython evaluation order:
* positional param annotations (subOrder 0, subIndex = argument
* position), `*args` annotation (1, 0), keyword-only annotations
* (2, position), `**kwargs` annotation (3, 0), return annotation
* (4, 0).
*/
private predicate functionAnnotation(
Py::FunctionExpr funcExpr, Py::Expr a, int subOrder, int subIndex
) {
a = funcExpr.getArgs().getAnnotation(subIndex) and subOrder = 0
or
a = funcExpr.getArgs().getVarargannotation() and subOrder = 1 and subIndex = 0
or
a = funcExpr.getArgs().getKwAnnotation(subIndex) and subOrder = 2
or
a = funcExpr.getArgs().getKwargannotation() and subOrder = 3 and subIndex = 0
or
a = funcExpr.getReturns() and subOrder = 4 and subIndex = 0
}

/** A lambda expression (has default args evaluated at definition time). */
additional class LambdaExpr extends Expr {
private Py::Lambda lambda;
Expand Down
4 changes: 2 additions & 2 deletions ...l/test/experimental/library-tests/CallGraph-type-annotations/InlineCallGraphTest.expected
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
testFailures
| type_annotations.py:6:16:6:32 | Comment # $ tt=Foo.method | Missing result: tt=Foo.method |
| type_annotations.py:16:16:16:32 | Comment # $ tt=Foo.method | Missing result: tt=Foo.method |
debug_callableNotUnique
pointsTo_found_typeTracker_notFound
typeTracker_found_pointsTo_notFound
| type_annotations.py:6:5:6:14 | Attribute() | Foo.method |
| type_annotations.py:16:5:16:14 | Attribute() | Foo.method |
| type_annotations.py:29:5:29:14 | Attribute() | Foo.method |
7 changes: 0 additions & 7 deletions python/ql/test/library-tests/frameworks/aiohttp/InlineTaintTest.expected
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,3 @@
argumentToEnsureNotTaintedNotMarkedAsSpurious
untaintedArgumentToEnsureTaintedNotMarkedAsMissing
| taint_test.py:151:9:151:15 | taint_test.py:151 | ERROR, you should add `# $ MISSING: tainted` annotation | request |
| taint_test.py:152:9:152:19 | taint_test.py:152 | ERROR, you should add `# $ MISSING: tainted` annotation | request.url |
| taint_test.py:153:9:153:36 | taint_test.py:153 | ERROR, you should add `# $ MISSING: tainted` annotation | Await |
testFailures
| taint_test.py:151:18:151:28 | Comment # $ tainted | Missing result: tainted |
| taint_test.py:152:22:152:32 | Comment # $ tainted | Missing result: tainted |
| taint_test.py:153:39:153:49 | Comment # $ tainted | Missing result: tainted |
| taint_test.py:168:76:168:96 | Comment # $ SPURIOUS: tainted | Fixed spurious result: tainted |
45 changes: 0 additions & 45 deletions python/ql/test/library-tests/frameworks/fastapi/ConceptsTest.expected
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,45 +0,0 @@
| response_test.py:11:30:11:37 | Parameter | Unexpected result: routedParameter=response |
| response_test.py:12:41:12:151 | Comment # $ CookieWrite CookieName="key" CookieValue="value" CookieSecure=false CookieHttpOnly=false CookieSameSite=Lax | Missing result: CookieHttpOnly=false |
| response_test.py:12:41:12:151 | Comment # $ CookieWrite CookieName="key" CookieValue="value" CookieSecure=false CookieHttpOnly=false CookieSameSite=Lax | Missing result: CookieName="key" |
| response_test.py:12:41:12:151 | Comment # $ CookieWrite CookieName="key" CookieValue="value" CookieSecure=false CookieHttpOnly=false CookieSameSite=Lax | Missing result: CookieSameSite=Lax |
| response_test.py:12:41:12:151 | Comment # $ CookieWrite CookieName="key" CookieValue="value" CookieSecure=false CookieHttpOnly=false CookieSameSite=Lax | Missing result: CookieSecure=false |
| response_test.py:12:41:12:151 | Comment # $ CookieWrite CookieName="key" CookieValue="value" CookieSecure=false CookieHttpOnly=false CookieSameSite=Lax | Missing result: CookieValue="value" |
| response_test.py:12:41:12:151 | Comment # $ CookieWrite CookieName="key" CookieValue="value" CookieSecure=false CookieHttpOnly=false CookieSameSite=Lax | Missing result: CookieWrite |
| response_test.py:13:51:13:161 | Comment # $ CookieWrite CookieName="key" CookieValue="value" CookieSecure=false CookieHttpOnly=false CookieSameSite=Lax | Missing result: CookieHttpOnly=false |
| response_test.py:13:51:13:161 | Comment # $ CookieWrite CookieName="key" CookieValue="value" CookieSecure=false CookieHttpOnly=false CookieSameSite=Lax | Missing result: CookieName="key" |
| response_test.py:13:51:13:161 | Comment # $ CookieWrite CookieName="key" CookieValue="value" CookieSecure=false CookieHttpOnly=false CookieSameSite=Lax | Missing result: CookieSameSite=Lax |
| response_test.py:13:51:13:161 | Comment # $ CookieWrite CookieName="key" CookieValue="value" CookieSecure=false CookieHttpOnly=false CookieSameSite=Lax | Missing result: CookieSecure=false |
| response_test.py:13:51:13:161 | Comment # $ CookieWrite CookieName="key" CookieValue="value" CookieSecure=false CookieHttpOnly=false CookieSameSite=Lax | Missing result: CookieValue="value" |
| response_test.py:13:51:13:161 | Comment # $ CookieWrite CookieName="key" CookieValue="value" CookieSecure=false CookieHttpOnly=false CookieSameSite=Lax | Missing result: CookieWrite |
| response_test.py:14:96:14:205 | Comment # $ CookieWrite CookieName="key" CookieValue="value" CookieSecure=false CookieHttpOnly=true CookieSameSite=Lax | Missing result: CookieHttpOnly=true |
| response_test.py:14:96:14:205 | Comment # $ CookieWrite CookieName="key" CookieValue="value" CookieSecure=false CookieHttpOnly=true CookieSameSite=Lax | Missing result: CookieName="key" |
| response_test.py:14:96:14:205 | Comment # $ CookieWrite CookieName="key" CookieValue="value" CookieSecure=false CookieHttpOnly=true CookieSameSite=Lax | Missing result: CookieSameSite=Lax |
| response_test.py:14:96:14:205 | Comment # $ CookieWrite CookieName="key" CookieValue="value" CookieSecure=false CookieHttpOnly=true CookieSameSite=Lax | Missing result: CookieSecure=false |
| response_test.py:14:96:14:205 | Comment # $ CookieWrite CookieName="key" CookieValue="value" CookieSecure=false CookieHttpOnly=true CookieSameSite=Lax | Missing result: CookieValue="value" |
| response_test.py:14:96:14:205 | Comment # $ CookieWrite CookieName="key" CookieValue="value" CookieSecure=false CookieHttpOnly=true CookieSameSite=Lax | Missing result: CookieWrite |
| response_test.py:15:58:15:221 | Comment # $ headerWriteName="Set-Cookie" headerWriteValue="key2=value2" CookieWrite CookieRawHeader="key2=value2" CookieSecure=false CookieHttpOnly=false CookieSameSite=Lax | Missing result: CookieHttpOnly=false |
| response_test.py:15:58:15:221 | Comment # $ headerWriteName="Set-Cookie" headerWriteValue="key2=value2" CookieWrite CookieRawHeader="key2=value2" CookieSecure=false CookieHttpOnly=false CookieSameSite=Lax | Missing result: CookieRawHeader="key2=value2" |
| response_test.py:15:58:15:221 | Comment # $ headerWriteName="Set-Cookie" headerWriteValue="key2=value2" CookieWrite CookieRawHeader="key2=value2" CookieSecure=false CookieHttpOnly=false CookieSameSite=Lax | Missing result: CookieSameSite=Lax |
| response_test.py:15:58:15:221 | Comment # $ headerWriteName="Set-Cookie" headerWriteValue="key2=value2" CookieWrite CookieRawHeader="key2=value2" CookieSecure=false CookieHttpOnly=false CookieSameSite=Lax | Missing result: CookieSecure=false |
| response_test.py:15:58:15:221 | Comment # $ headerWriteName="Set-Cookie" headerWriteValue="key2=value2" CookieWrite CookieRawHeader="key2=value2" CookieSecure=false CookieHttpOnly=false CookieSameSite=Lax | Missing result: CookieWrite |
| response_test.py:15:58:15:221 | Comment # $ headerWriteName="Set-Cookie" headerWriteValue="key2=value2" CookieWrite CookieRawHeader="key2=value2" CookieSecure=false CookieHttpOnly=false CookieSameSite=Lax | Missing result: headerWriteName="Set-Cookie" |
| response_test.py:15:58:15:221 | Comment # $ headerWriteName="Set-Cookie" headerWriteValue="key2=value2" CookieWrite CookieRawHeader="key2=value2" CookieSecure=false CookieHttpOnly=false CookieSameSite=Lax | Missing result: headerWriteValue="key2=value2" |
| response_test.py:16:68:16:231 | Comment # $ headerWriteName="Set-Cookie" headerWriteValue="key2=value2" CookieWrite CookieRawHeader="key2=value2" CookieSecure=false CookieHttpOnly=false CookieSameSite=Lax | Missing result: CookieHttpOnly=false |
| response_test.py:16:68:16:231 | Comment # $ headerWriteName="Set-Cookie" headerWriteValue="key2=value2" CookieWrite CookieRawHeader="key2=value2" CookieSecure=false CookieHttpOnly=false CookieSameSite=Lax | Missing result: CookieRawHeader="key2=value2" |
| response_test.py:16:68:16:231 | Comment # $ headerWriteName="Set-Cookie" headerWriteValue="key2=value2" CookieWrite CookieRawHeader="key2=value2" CookieSecure=false CookieHttpOnly=false CookieSameSite=Lax | Missing result: CookieSameSite=Lax |
| response_test.py:16:68:16:231 | Comment # $ headerWriteName="Set-Cookie" headerWriteValue="key2=value2" CookieWrite CookieRawHeader="key2=value2" CookieSecure=false CookieHttpOnly=false CookieSameSite=Lax | Missing result: CookieSecure=false |
| response_test.py:16:68:16:231 | Comment # $ headerWriteName="Set-Cookie" headerWriteValue="key2=value2" CookieWrite CookieRawHeader="key2=value2" CookieSecure=false CookieHttpOnly=false CookieSameSite=Lax | Missing result: CookieWrite |
| response_test.py:16:68:16:231 | Comment # $ headerWriteName="Set-Cookie" headerWriteValue="key2=value2" CookieWrite CookieRawHeader="key2=value2" CookieSecure=false CookieHttpOnly=false CookieSameSite=Lax | Missing result: headerWriteName="Set-Cookie" |
| response_test.py:16:68:16:231 | Comment # $ headerWriteName="Set-Cookie" headerWriteValue="key2=value2" CookieWrite CookieRawHeader="key2=value2" CookieSecure=false CookieHttpOnly=false CookieSameSite=Lax | Missing result: headerWriteValue="key2=value2" |
| response_test.py:17:53:17:116 | Comment # $ headerWriteName="X-MyHeader" headerWriteValue="header-value" | Missing result: headerWriteName="X-MyHeader" |
| response_test.py:17:53:17:116 | Comment # $ headerWriteName="X-MyHeader" headerWriteValue="header-value" | Missing result: headerWriteValue="header-value" |
| response_test.py:23:26:23:29 | Parameter | Unexpected result: routedParameter=resp |
| response_test.py:41:42:41:49 | Parameter | Unexpected result: routedParameter=response |
| response_test.py:48:41:48:151 | Comment # $ CookieWrite CookieName="key" CookieValue="value" CookieSecure=false CookieHttpOnly=false CookieSameSite=Lax | Missing result: CookieHttpOnly=false |
| response_test.py:48:41:48:151 | Comment # $ CookieWrite CookieName="key" CookieValue="value" CookieSecure=false CookieHttpOnly=false CookieSameSite=Lax | Missing result: CookieName="key" |
| response_test.py:48:41:48:151 | Comment # $ CookieWrite CookieName="key" CookieValue="value" CookieSecure=false CookieHttpOnly=false CookieSameSite=Lax | Missing result: CookieSameSite=Lax |
| response_test.py:48:41:48:151 | Comment # $ CookieWrite CookieName="key" CookieValue="value" CookieSecure=false CookieHttpOnly=false CookieSameSite=Lax | Missing result: CookieSecure=false |
| response_test.py:48:41:48:151 | Comment # $ CookieWrite CookieName="key" CookieValue="value" CookieSecure=false CookieHttpOnly=false CookieSameSite=Lax | Missing result: CookieValue="value" |
| response_test.py:48:41:48:151 | Comment # $ CookieWrite CookieName="key" CookieValue="value" CookieSecure=false CookieHttpOnly=false CookieSameSite=Lax | Missing result: CookieWrite |
| response_test.py:49:87:49:184 | Comment # $ headerWriteName="Custom-Response-Type" headerWriteValue="yes, but only after function has run" | Missing result: headerWriteName="Custom-Response-Type" |
| response_test.py:49:87:49:184 | Comment # $ headerWriteName="Custom-Response-Type" headerWriteValue="yes, but only after function has run" | Missing result: headerWriteValue="yes, but only after function has run" |
Loading
Loading