mini_buildd.httpd module

Error

twisted < 24 + SSL (mbd_workaround_ssl): Sporadic errors reading event queue

Reproduce: ./devel profile _ssl updatetestall eventually yields to some error reading events (nearly every time).

With mbd_workaround_ssl in place, empirical data seems to suggest less errors, so this is still enabled for older versions.

Extended empirical testing shows that all these problems are really gone with twisted 24.

Error

twisted < 23.10.0 (mbd_workaround_producer): Twisted can’t serve APT repository when HTTP pipelining is enabled (https://github.com/twisted/twisted/issues/11976)

Seemingly random:

RuntimeError: Cannot register producer <twisted.web.static.NoRangeStaticProducer object at 0xfoo>, because producer <twisted.internet._producer_helpers._PullToPush object at 0xbar> was never unregistered.

errors from twisted. This in turn randomly breaks apt update calls with (slightly misleading) ‘size mismatch’ errors.

This error appears when APT has HTTP pipelining enabled (which it has by default) and reasonable big indices files. You may repeat this bug in plain twisted if you serve any reasonably sized repository like so:

cd <your_repo_dir>
twistd3 --nodaemon --pidfile /tmp/twistd.pid web --path $(pwd)
..
EDIT /etc/apt/sources.list.d/test.list                           # Add resp. APT line
sudo mini-buildd-internals sbuild-setup-blocks apt-clear --run   # Be sure error is not hidden by cached apt lines
sudo apt update

With mbd_workaround_producer in place makes APT pipelining work, so it’s still in place for older versions. Fwiw, the worakround however most likely causes these seemingly random (but not practically breaking mini-buildd) errors:

builtins.AttributeError: 'NoneType' object has no attribute 'unregisterProducer'
class mini_buildd.httpd.Site(resource, requestFactory=None, *args, **kwargs)

Bases: Site

class mini_buildd.httpd.RootResource(wsgi_resource)

Bases: Resource

Twisted root resource needed to mix static and wsgi resources

getChild(path, request)

Retrieve a ‘child’ resource from me.

Implement this to create dynamic resource generation – resources which are always available may be registered with self.putChild().

This will not be called if the class-level variable ‘isLeaf’ is set in your subclass; instead, the ‘postpath’ attribute of the request will be left as a list of the remaining path elements.

For example, the URL /foo/bar/baz will normally be:

| site.resource.getChild('foo').getChild('bar').getChild('baz').

However, if the resource returned by ‘bar’ has isLeaf set to true, then the getChild call will never be made on it.

Parameters and return value have the same meaning and requirements as those defined by L{IResource.getChildWithDefault}.

mini_buildd.httpd.mbd_is_ssl(request)

For workarounds only. Does not work with twisted 24

class mini_buildd.httpd.FileResource(path: str, defaultType: str = 'text/html', ignoredExts: Sequence[str] = (), registry: Registry | None = None, allowExt: Literal[0] = 0)

Bases: File

Twisted static resource

NEEDS_PRODUCER_WORKAROUND = False
classmethod mbd_workaround_producer(request)
render(request)

Render a given resource. See L{IResource}’s render method.

I delegate to methods of self with the form ‘render_METHOD’ where METHOD is the HTTP that was used to make the request. Examples: render_GET, render_HEAD, render_POST, and so on. Generally you should implement those methods instead of overriding this one.

render_METHOD methods are expected to return a byte string which will be the rendered page, unless the return value is C{server.NOT_DONE_YET}, in which case it is this class’s responsibility to write the results using C{request.write(data)} and then call C{request.finish()}.

Old code that overrides render() directly is likewise expected to return a byte string or NOT_DONE_YET.

@see: L{IResource.render}

class mini_buildd.httpd.Events

Bases: Resource

NEEDS_SSL_WORKAROUND = False
classmethod mbd_workaround_ssl(request)
render_GET(request)
class mini_buildd.httpd.HttpD(wsgi_app, minthreads=0, maxthreads=10)

Bases: Thread

shutdown()
mbd_run()