# Known Issues

Also check Airtime's / LibreTime's [troubleshooting page](https://libretime.org/manual/troubleshooting/).

## Known log file locations

`/var/log/airtime/pypo-liquidsoap`\
`/var/log/airtime/pypo/notify.log`\
`/var/log/airtime/zendlog.log`

### Status

The [status page in Airtime](http://studio.radioangrezi.de/systemstatus) does not show a status for some services: `airtime_analyzer`, `airtime-playout`, `airtime-liquidsoap`. The services might be running correctly though. You can check via our [own monitoring](http://studio.radioangrezi.de:2812/) or `service SERVICE-NAME status` and `journalctl -u SERVICE-NAME | tail`

## Error 500 / No PHP logs

You would suspect PHP logs in `/var/log/airtime/zendlog.log`. But there is none (or it is not up to date). Check `/var/log/apache2/error.log`: It might show:

```
[Mon Dec 16 16:36:41.959122 2019] [:error] [pid 26398] [client 127.0.0.1:39688] PHP Fatal error:  Uncaught exception 'Zend_Log_Exception' with message '"/var/log/airtime/zendphp.log" cannot be opened with mode "a"' in /usr/share/airtime/php/vendor/zendframework/zendframework1/library/Zend/Log/Writer/Stream.php:81\nStack trace:\n#0 /usr/share/airtime/php/airtime_mvc/application/logging/Logging.php(11): Zend_Log_Writer_Stream->__construct('/var/log/airtim...')\n#1 /usr/share/airtime/php/airtime_mvc/application/logging/Logging.php(83): Logging::getLogger()\n#2 /usr/share/airtime/php/airtime_mvc/application/logging/Logging.php(166): Logging::error('Uncaught except...')\n#3 [internal function]: Logging::loggingShutdownCallback()\n#4 {main}\n  thrown in /usr/share/airtime/php/vendor/zendframework/zendframework1/library/Zend/Log/Writer/Stream.php on line 81
```

Check the permissions on `/var/log/airtime/zendlog.log`. It should be 0777, probably.

## DNS Problems

2019-05-20

Recently the HfK-network experienced issues with its DNS Servers (172.16.0.1 + 172.16.0.2). Local streaming services (darkice, airtime, icecast) were affecetd. Resolved by adding additional 3rd-party DNS Servers form Digitalcourage (46.182.19.48) + Chaos Computer Club (213.73.91.35) to /etc/netplan/XX.yaml.

## RabbitMQ hostname changes

2019-05-31

form: <https://libretime.org/manual/troubleshooting/>

If the Airtime logs indicate failures to connect to the RabbitMQ server, such as:

```
2013-10-31 08:21:11,255 ERROR - [pypomessagehandler.py : main() : line 
99] - Error connecting to RabbitMQ Server. Trying again in few seconds
2013-10-31 08:21:11,255 ERROR - [pypomessagehandler.py : main() : line 99] - Error connecting to RabbitMQ Server. Trying again in few seconds - See more at: http://forum.sourcefabric.org/discussion/16050/#sthash.W8OJrNFm.dpuf
```

but the RabbitMQ server is running normally, this error might be due to a change in the server's hostname since Airtime installation. Directory names under /var/lib/rabbitmq/mnesia/ indicate that RabbitMQ's database files are organised according to the hostname of the server, for example:

```
rabbit@airtime
```

where the hostname is airtime.example.com. If the hostname has changed, it may be necessary to reconfigure RabbitMQ manually, as follows:

Delete the files in /var/lib/rabbitmq/mnesia/

```
sudo rm -r /var/lib/rabbitmq/mnesia/*
```

Restart RabbitMQ:

```
sudo invoke-rc.d rabbitmq-server restart
```

Enter the following commands to set up authentication and grant permissions. The rabbitmqctl add\_user command requires the RabbitMQ password from the /etc/airtime/airtime.conf file as an argument. The rabbitmqctl set\_permissions command should be entered on one line, with the list of Airtime services repeated three times:

```
rabbitmqctl add_vhost /airtime rabbitmqctl add_user airtime XXXXXXXXXXXXXXXXXXXX
rabbitmqctl set_permissions -p /airtime airtime "airtime-pypo|pypo-fetch|airtime-analyzer|media-monitor"   "airtime-pypo|pypo-fetch|airtime-analyzer|media-monitor"  "airtime-pypo|pypo-fetch|airtime-analyzer|media-monitor"
```

## PHP Errors

2019-06-01

Libretime logs PHP errors into `/var/log/airtime/zendphp.log` You can use Libretimes log controller to add custom log statements to the code. (Native `echo()` and `print_r()` will not help much.) Use `Logging::error()`, `Logging::warn()`, `Logging::info()` or `Logging::debug()` from anywhere in the Libretime codebase.

## Error 500 in airtime-playout (pypo) and `Invalid argument supplied for foreach()` zendphp.log

2019-06-21

### Phenomenon

airtime-playout (`sudo -u www-data python /usr/bin/airtime-playout`) is outputting 500 errors for all API requests:

```
2019-06-01 19:36:06,106 [listenerstat] [ERROR]  Exception: HTTP Error 500: Internal Server Error
2019-06-01 19:38:06,323 [api_client] [ERROR]  HTTP Error 500: Internal Server Error
2019-06-01 19:38:06,373 [api_client] [ERROR]  HTTP Error 500: Internal Server Error
```

zendphp.log is showing errors like:

```
2019-06-01T19:36:06+00:00 ERR (3): studio.radioangrezi.de [ErrorController.php:54 - errorAction()] - An internal application error has occurred.: exception 'ErrorException' with message 'Invalid argument supplied for foreach()' in /usr/share/airtime/php/airtime_mvc/application/models/ListenerStat.php:98
Stack trace:
#0 [internal function]: exception_error_handler(2, 'Invalid argumen...', '/usr/share/airt...', 98, Array)
#1 /usr/share/airtime/php/vendor/zendframework/zendframework1/library/Zend/Log.php(636): call_user_func('exception_error...', 2, 'Invalid argumen...', '/usr/share/airt...', 98, Array)
#2 /usr/share/airtime/php/airtime_mvc/application/models/ListenerStat.php(98): Zend_Log->errorHandler(2, 'Invalid argumen...', '/usr/share/airt...', 98, Array)
#3 /usr/share/airtime/php/airtime_mvc/application/controllers/ApiController.php(1273): Application_Model_ListenerStat::insertDataPoints(NULL)
#4 /usr/share/airtime/php/vendor/zendframework/zendframework1/library/Zend/Controller/Action.php(516): ApiController->pushStreamStatsAction()
#5 /usr/share/airtime/php/vendor/zendframework/zendframework1/library/Zend/Controller/Dispatcher/Standard.php(308): Zend_Controller_Action->dispatch('pushStreamStats...')
#6 /usr/share/airtime/php/vendor/zendframework/zendframework1/library/Zend/Controller/Front.php(954): Zend_Controller_Dispatcher_Standard->dispatch(Object(Zend_Controller_Request_Http), Object(Zend_Controller_Response_Http))
#7 /usr/share/airtime/php/vendor/zendframework/zendframework1/library/Zend/Application/Bootstrap/Bootstrap.php(105): Zend_Controller_Front->dispatch()
#8 /usr/share/airtime/php/vendor/zendframework/zendframework1/library/Zend/Application.php(384): Zend_Application_Bootstrap_Bootstrap->run()
#9 /usr/share/airtime/php/airtime_mvc/application/airtime-boot.php(84): Zend_Application->run()
#10 /usr/share/airtime/php/airtime_mvc/public/index.php(68): require_once('/usr/share/airt...')
#11 {main}
`
```

for `ListenerStat.php:98` and `ApiController.php:1870`

### Not-the-solution

* Checked files: `ListenerStat.php` and `ApiController.php` -> POST getParams() always empty. -> no data -> null / true object (which is not iterable).
* Also checked: `airtime-playout.py` / `listenerstat.py` and `api_client.py` -> all not faulty, because dumped (tcpdump) http packets showed correct payload.
* Notices on the side, but not related: `SQLSTATE[55P03]: Lock not available: 7 ERROR: could not obtain lock on row in relation "cc_pref"` ([#465](https://github.com/LibreTime/libretime/issues/465))
* in place updated of Airtime / Libretime will not be easy. ([#623](https://github.com/LibreTime/libretime/issues/623))

### Solution

The hostname and default apache vhost was changed. Since than the error occured. Apache was 301 forwarding requests made to `anregzistudio.hfk-bremen.de` to `studio.radioangrezi.de`. POST data is lost when forwarding!

Put correct hostname in `base_url` of `/etc/airtime.conf`. This would go for a port change as well (e.g. HTTPS)!

```
  3 [general]
  4 api_key = ...
  5 web_server_user = www-data
  6 base_url = studio.radioangrezi.de
  7 base_port = 80
```

### Helpful

* Dump local http requests with payload: `sudo tcpdump -s 0 -A 'tcp dst port 80 and (tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x504f5354)' -i lo`
* ["Looks like your POST data get lost when redirecting"](https://stackoverflow.com/a/3654142)
* [Very similar issue: Wrong port, not wrong hostname.](https://github.com/LibreTime/libretime/issues/749)
* [Also simmilar: Wrong hostname in /etc/hosts](https://github.com/LibreTime/libretime/issues/299)

## "Getting information from the server..."

Airtime shows only "Getting information from the server..." on the [stream preferences page](http://studio.radioangrezi.de/preference/stream-setting).&#x20;

```
Traceback (most recent call last):
   File "/usr/bin/pyponotify", line 4, in <module>
     __import__('pkg_resources').run_script('airtime-playout==1.0', 'pyponotify')
   File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 664, in run_script
     self.require(requires)[0].run_script(script_name, ns)
   File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 1444, in run_script
     exec(code, namespace, namespace)
   File "/usr/local/lib/python2.7/dist-packages/airtime_playout-1.0-py2.7.egg/EGG-INFO/scripts/pyponotify", line 6
     backupCount=8)
   File "/usr/lib/python2.7/logging/handlers.py", line 117, in __init__
     BaseRotatingHandler.__init__(self, filename, mode, encoding, delay)
   File "/usr/lib/python2.7/logging/handlers.py", line 64, in __init__
     logging.FileHandler.__init__(self, filename, mode, encoding, delay)
   File "/usr/lib/python2.7/logging/__init__.py", line 920, in __init__
     StreamHandler.__init__(self, self._open())
   File "/usr/lib/python2.7/logging/__init__.py", line 950, in _open
     stream = open(self.baseFilename, self.mode)
 IOError: [Errno 2] No such file or directory: '/var/log/airtime/pypo/notify.log'

```

**Resolution**: Log files could not be written. Add log directory and give write permissions.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.radioangrezi.de/tech-deep-dive/known-issues.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
