Commit Graph

97 Commits (edc77236c708ca248f0c9b2139bf7e2a49dbf736)

Author SHA1 Message Date
Uku Taht 2cf0c1a456 Use `forwarded` header when X-Forwarded-For is missing 6 years ago
Vignesh Joglekar 41e4690116
Adds Time on Page metric to Top Pages report (#1007)
* First pass

Needs more testing & potentially cleanup

* Fixes tests, error handling

* Formatting

* Removes broken test

* Fixes inconsistent test

This was due to Clickhouse setup not inserting the sessions with the exact same timestamp consistently and making the test inconsistent

* Combines `include` param, asyncs time_on_page and bounce_rate

* Fixes CH error when no pageviews exist in period

* Format

* Changelog

* Increases await timeout to accomodate larger data sets

* Improves handling of timeout behavior

* Fixes session-based filtering on time on page queries

* Formatting

* Removes old forced entry page modal from top pages report
6 years ago
Uku Taht 7f9057818d Protect the API endpoint from creating too many sites 6 years ago
Uku Taht 5efa15f381 Use CF-Connecting-IP when present for remote IP address 6 years ago
Uku Taht 0799138622 Add nil check to hashing 6 years ago
Uku Taht 5c77ec3f5d
Support multiple `data-domain` values (#968)
* Support multiple `data-domain` values

* Add changelog entry
6 years ago
Uku Taht b107717774
Additional metrics for timeseries API call and CSV export (#952)
* Add `metrics` option to timeseries call

* Add more metrics to csv export

* Add changelog entry
6 years ago
Uku Taht 86d73b68b8 Fix formatting 6 years ago
Uku Taht 70f0657327 Update shared link API 6 years ago
Uku Taht 956e92f161 Change default timezone to Etc/UTC 6 years ago
Uku Taht 80abfa6ef6 Add validations 6 years ago
Uku Taht d473accf40 Add API key scopes 6 years ago
Uku Taht 3ed964b45b Add API endpoints for site and shared link creation 6 years ago
Uku Taht c4c0bb2095 Various formatting improvements
* Format numbers over 1B
* Show full number when hovering
6 years ago
Uku Taht 3eb2f6d7a8
Add support for sampling (#875)
* Add support for sampling

* Use correct mix package

* Add notice about data sampling
6 years ago
Uku Taht 432ad2a606
Stats API improvements (#826)
* Show hourly data for a day

* Fix aggregations with long periods and comaprisons enabled

* Return Direct / None instead of empty string when missing source

* Fix/Align breakdown query with current queries in the UI

* Show correct number of days for 7d and 30d

* Wrap results of aggregate call in `results`

* Allow breakdown by event:name

* Ensure event:name filter works in all contexts

* WIP

* Add support for custom event props
6 years ago
Uku Taht 0ceb539a22 Reduce errors sent to Sentry 6 years ago
Vignesh Joglekar b952053cee
Adds same glob patterns as exclusions for pageview goals (#750)
* First Pass - 90% wrong

* Second pass - basically done

* Swaps N+1 for multiMatchAllIndices & friends

* New version of query with just one regex computation

* Slight formatting, basic test

* Updates tests to match master new events

* Changelog

* Adds more sophisticated glob tests
6 years ago
Vignesh Joglekar ff32218bd0
Adds entry and exit pages to Top Pages module (#712)
* Initial Pass

* Adds support for page visits counting by referrer

* Includes goal selection in entry and exit computation

* Adds goal-based entry and exit page stats, formatting, code cleanup

* Changelog

* Format

* Exit rate, visit duration, updated tests

* I keep forgetting to format :/

* Tests, last time

* Fixes double counting, exit rate >100%, relevant tests

* Fixes exit pages on filter and goal states

* Adds entry and exit filters, fixes various bugs

* Fixes discussed issues

* Format

* Fixes impossible case in tests

Originally, there were only 2 pageviews for `test-site.com`,`/` on `2019-01-01`, but that doesn't make sense when there were 3 sessions that exited on the same site/date.

* Format

* Removes boolean function parameter in favor of separate function

* Adds support for queries that use `page` filter as `entry-page`

* Format

* Makes  loader/title interaction in sources report consistent
6 years ago
Uku Taht 3c4931598b Fix breakdown call 6 years ago
Uku Taht a29ac44434
Api improvements (#723)
* Require date parameter with custom period

* Validate format of the `date` param with custom period

* Return proper status codes for auth failures

* Add breakdown endpoint

* Fix pagination

* Add API validations for breakdown call

* Change the breakdown endpoint to return an object instead of an array

* Remove change to aggregate call

* Change timeseries call
6 years ago
Uku Taht 77628d04c9
Validate API query params (#703)
* Add validations for `date`, `period` and `metrics` variables

* Validate interval query parameter
6 years ago
Uku Taht 5acb5b7039
Stats API (#679)
* WIP

* Add ability to filter by anything

* Add API keys

* Add version to api endpoint

* Fix API test route

* Fix API tests

* Allow 'date' parameter in '6mo' and '12mo'

* Rename session -> visit in API filters

* Filter expressions in the API

* Implement filters in aggregate call

* Add `compare` option to aggregate call

* Add way to manage API keys through the UI

* Authenticate with API key

* Use API key in tests
6 years ago
Uku Taht 81c12884cd
Add elixir action (#526)
* Add elixir action

* Format the codebase

* Add postgresql

* Postgres config

* Run postgres on localhost

* Add clickhouse to CI
6 years ago
Uku Taht ef4af9249f Fix double quote handling in Clickhouse 6 years ago
Uku Taht 19a9e314fc Accept anything in prop value as long as it can `to_string` 6 years ago
Uku Taht 1d4bddd79d Decode URLs properly 6 years ago
Uku Taht b0b5d92f6e Fix URL decoding 6 years ago
Uku Taht d0d7b823f8
Browser and OS version (#397)
* Collect browser and OS version

* Display browser version

* Show operating system versions

* Device categorization

* Treat headless chrome like a bot

* Ignore events from automated browsers

* Only take major and minor of the version

* Add tests

* Add CHANGELOG entry

* Add changelog entry for bots

* Store empty value as browser when unknown
6 years ago
Uku Taht 0a7684f3bc Mix format 6 years ago
Uku Taht b816d92020 Show conversion rates in conversions report 6 years ago
Uku Taht 044da18fec Fix tests for metadata 6 years ago
Uku Taht 40900c7653
Ability to add event metadata (#381)
* Ability to add event metadata

* Close Dropdown on outside click

* Show (none) value in metadata breakdown

* Allow filtering for metadata key/val pairs

* Use correct clickhouse_ecto

* Better naming for meta filter

* Add tests

* Add changelog entry

* Remove change made for testing
6 years ago
Uku Taht e228fd1f85
Additional filters (#358)
* Filter by screen size

* Filter by browsers

* Add filter for Operating system

* Add changelog entry

* Add filter for countries

* Add tests for filters
6 years ago
Uku Taht 8d9667a949
Show utm_medium, utm_source, and utm_campaign in sources modal (#321)
* Show utm_medium, utm_source, and utm_campaign in sources modal

* Allow filtering by UTM tags

* Integrate filters with URL bar

* Add CHANGELOG entry

* Refresh modal when filter changes

* Remove Direct / None from campaign results

* Add UTM tabs to top sources report

* Add pagination

* Remove dropdown from top sources popup

* Fix bug in clickhouse_ecto

* Remove referrers_for_goal

* Make sure UTM tags work OK with goals

* Make source tab selection sticky

* Consistent styling for devices and source tabs

* Add noref in realtime to utm tabs

* Fix tests
6 years ago
Uku Taht 122dd94327
Store `utm_medium`, `utm_source`, and `utm_campaign` (#318) 6 years ago
Uku Taht 4b36bb7138
Use clickhouse_ecto for db connection (#317)
* Use clickhouse-ecto for stats

* Use clickhouse ecto instead of low-level clickhousex

* Remove defaults from event schema

* Remove all references to Clickhousex

* Document configuration change

* Ensure createdb and migrations can be run in a release

* Remove config added for debug

* Update plausible_variables.sample.env
6 years ago
Uku Taht 49f153c380
Paginate referrers (#313)
* Add pagination for referrers

* Fix top referrers for goal
6 years ago
Uku Taht 4aa4dfdcaf
Hash mode (#299)
* Build tracker with hash mode

* Extract hash fragment in hash mode

* Serve new hash-based tracker
6 years ago
Uku Taht d30f17c807
Extract utm_source on the server side (#298)
* Extract utm_source on the server side

* Build tracker scripts
6 years ago
Uku Taht c200b95ce0 Add test for shorthands in external API 6 years ago
Uku Taht 76896f7cee
Show total conversions in addition to uniques (#294) 6 years ago
Uku Taht 01ef17abb0
Make sure all future URLs and sources are decoded (#292) 6 years ago
Uku Taht 503dc60b30 Rename Total visitors -> Unique visitors 6 years ago
Uku Taht ff64fccb18
Add site switcher (#281)
* WIP

* Add site switcher

* Run ci

* Highlight active site

* Do not show sites dropdown for logged out users

* Add site switcher to realtime view

* Remove blue border when logged out
6 years ago
Uku Taht 0a2a2656aa
Page drilldown (#276)
* WIP

* Add filter option for page

* Use JOIN to get correct top numbers

* Filter google search terms by page

* Allow filtering by page from modal

* More robust base url for google search console integration

* Fix tests
6 years ago
Uku Taht f781c5647c
Direct traffic improvements (#268)
* Direct traffic improvements

* Make sure an empty icon shows always

* Enable direct traffic drilldown
6 years ago
Uku Taht 99bc20a8b3
Referrer drilldown (#252)
* Referrer drilldown

* Render external links for referrer drilldown modal

* Rename top pages to entry pages

* Add google keywords to referrer drilldown

* Fix link in source modal

* Remove back link from referrer drilldown

* Fix tests

* Mix format

* Remove unused function

* Get top stats from the events table

* Revert "Get top stats from the events table"

This reverts commit 67384e2d35.

* Fix pages modal title

* Improve pages report in referrer drilldown

* Fix label in referrer drilldown
6 years ago
Chandra Tungathurthi f7b37fe9ea
Selhosted version Improvements and additional features (#209)
* first commit with test and compile job

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding 'prepare' stage

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated ci script to include "test" compile phase

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding environment variables for connecting to postgresql

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated ci config for postgres

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* using non-alpine version of elixir

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* re-using the 'compile' artifacts and added explict env variables for testing

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* removing redundant deps fetching from common code

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* formatting using mix.format -- beware no-code changes!

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* added release config

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding consistent env variable for Database

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* more cleaning up of environment variables

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Adding releases config for enabling releases

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* cleaning up env configs

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Cleaned up config and prepared config for releases

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated CI script with new config for test

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Added Dockerfile for creating production docker image

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Adding "docker" build job yay!

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* using non-slim version of debian and installing webpack

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Adding overlays for migrations on releases

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* restricting the docker built to master branch only

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* typo fix

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding "Hosting.md" to explain hosting instructions

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* removed the default comments

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Added documentation related to env variables

* updated documentation and fixed typo

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated documentation

* Bumping up elixir version as `overlays` are only supported in latest version

read release notes: https://github.com/elixir-lang/elixir/releases/tag/v1.10.0

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Adding tarball assembly during release

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated HOSTING.md

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Added support for db migration

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* minor corrections

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* initializing admin user

Admin user has been added in the "migration" phase. A default user is automatically created in the process. One can provide the related env variables, else a new one will be automatically created for you.

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Initial base domain update - phase#1

These changes are only meant for correct operating it under self-hosting. There are many other cosmetic changes, that require updates to email, site and other places where the original website and author is used.

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Using dedicated config variable `base_domain` instead

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding base_domain to releases config

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* removing the dedicated config "base_domain", relying on endpoint host

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Removed the usage of "Mix" in code!

It is bad practice to use "mix" module inside the code as in actual release this module is unavailable. Replacing this with a config environment variable

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Added support for SMTP via Bamboo Smtp Adapter

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Capturing SMTP errors via Sentry

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Minor updates

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Adding junit formatter -- useful for generating test reports

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding documentation for default user

* Resolve "Gitlab Adoption: Add supported services in "Security & Compliance""

* bumping up the debian version to fix issues

fixing some vulnerabilities identified by the scanning tools

* More updates for self-hosting

Changes in most of the places to suit self-hosting. Although, there are some which have been left-off.

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* quick-dirty-fix!

* bumping up the db connect timeout

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* bumping up the db connect timeout

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* bumping up the db connect timeout

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* bumping up timeout - skipping MRs :-/

* removing restrictions on watching for changes

this stuff isn't working

* Update HOSTING.md

* renamed the module name

* reverting formatting-whitespace changes

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* reverting the name to release

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding docker-compose.yml and related instructions

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* using `plausible_url` instead of assuming `https`

this is because, it is much to test in local dev machines and in most cases there's already a layer above which is capable for `https` termination and http -> https upgrade

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* WIP: merging changes from upstream

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* wip: more changes

* Pushing in changes from upstream

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* changes to ci for testing

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* cleaning up and finishing clickhouse integration

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updating readme with hosting details

* removing deleted files from upstream

* minor config adjustments

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* formatting changes

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* changing the connection strategy for clickhouse during release

since clickhouse integration doesn't have an ecto support, we need to prepare the db _before_ the clickhouse migration. One workaround is to connect to a default db on init and then create a db

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* formatting

* cleanup and added separated migration to setup

* Big improvements to selfhosting

- added ability for disabling
  - authentication completely
  - registration
  - landing page

- formatting cleanups

* Big improvements to selfhosting

- added ability for disabling
  - authentication completely
  - registration
  - landing page

- formatting cleanups

* changing smtp auth  to optional

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* removed stale templates and permanently removed landing page

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* removed stale templates and permanently removed landing page

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* removed stale templates and permanently removed landing page

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* WIP

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* fixes form upstream merge

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* added disabling subscription for selfhosted version

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated doc

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* formatting

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Remove reference to file that doesn't exist

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* do not show direct traffic if there's no data

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* addressing PR comments

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* formatting

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
6 years ago
Uku Taht b254e943bc Add unique visitors to pages modal 6 years ago