Benson Wong 01d4838fb3 Fix token metrics parsing (#199) ...

Fix #198

- use llama-server's `timings` info if available in response body
- send "-1" for token/sec when not able to accurately calculate
  performance
- optimize streaming body search for metrics information

2025-07-22 23:10:14 -07:00

..

.gitignore

New UI (#157, #164)

2025-06-16 16:45:19 -07:00

config_posix_test.go

Add metrics logging for chat completion requests (#195)

2025-07-21 22:19:55 -07:00

config_test.go

Strip comments before macro expansion in config (#193)

2025-07-15 10:14:16 -07:00

config_windows_test.go

Add metrics logging for chat completion requests (#195)

2025-07-21 22:19:55 -07:00

config.go

Add metrics logging for chat completion requests (#195)

2025-07-21 22:19:55 -07:00

events.go

Add metrics logging for chat completion requests (#195)

2025-07-21 22:19:55 -07:00

helpers_test.go

Refactor all default config values into config.go (#162)

2025-06-15 12:32:00 -07:00

logMonitor_test.go

Add Event Bus (#184)

2025-07-01 22:17:35 -07:00

logMonitor.go

Fix High CPU (#189)

2025-07-15 18:04:30 -07:00

metrics_middleware.go

Fix token metrics parsing (#199)

2025-07-22 23:10:14 -07:00

metrics_monitor.go

Add metrics logging for chat completion requests (#195)

2025-07-21 22:19:55 -07:00

process_test.go

Strip comments before macro expansion in config (#193)

2025-07-15 10:14:16 -07:00

process.go

Fix High CPU (#189)

2025-07-15 18:04:30 -07:00

processgroup_test.go

Change /unload to not wait for inflight requests (#125)

2025-05-13 11:39:19 -07:00

processgroup.go

Change /unload to not wait for inflight requests (#125)

2025-05-13 11:39:19 -07:00

proxymanager_api.go

Add metrics logging for chat completion requests (#195)

2025-07-21 22:19:55 -07:00

proxymanager_loghandlers.go

Add Event Bus (#184)

2025-07-01 22:17:35 -07:00

proxymanager_test.go

Fix token metrics parsing (#199)

2025-07-22 23:10:14 -07:00

proxymanager.go

add guard to ensure ls-real-model-name is set in context

2025-07-21 22:59:41 -07:00

sanitize_cors_test.go

Sanitize CORS headers (#85)

2025-04-01 08:43:53 -07:00

sanitize_cors.go

Sanitize CORS headers (#85)

2025-04-01 08:43:53 -07:00

ui_embed.go

New UI (#157, #164)

2025-06-16 16:45:19 -07:00