Benson Wong e250e71e59 Include metrics from upstream chat requests (#361) ...

* proxy: refactor metrics recording

- remove metrics_middleware.go as this wrapper is no longer needed. This
  also eliminiates double body parsing for the modelID
- move metrics parsing to be part of MetricsMonitor
- refactor how metrics are recording in ProxyManager
- add MetricsMonitor tests
- improve mem efficiency of processStreamingResponse
- add benchmarks for MetricsMonitor.addMetrics
- proxy: refactor MetricsMonitor to be more safe handling errors

2025-10-25 17:38:18 -07:00

7.3 KiB

Raw Permalink Blame History

View Raw