Commit Graph

2606 Commits (fe0edb2e1e58a40841796f5616ff528bfaa8a196)
 

Author SHA1 Message Date
Karl-Aksel Puulmann fe0edb2e1e Fix test naming 3 years ago
Karl-Aksel Puulmann 66b7d383d2 Unify behavior of entry/exit page hostnames with rest 3 years ago
Karl-Aksel Puulmann 54867fb05d Merge remote-tracking branch 'origin/master' into new-table-querying 3 years ago
hq1 6af80dd246
Filter by hostnames (#3963)
* CH Migration: exit/entry hostnames in sessions_v2

* Leave only exit_page_hostname, we already record hostnames

* Use ClickHouse DDL in favour of ecto so that cluster is included

* Compress with ZSTD(3)

* Expose Hostname filter in the dashboard dropdown

* Add `exit_page_hostname` to ClickHouse `sessions_v2` schema

* Start tracking hostname changes in sessions

* Implement hostname filter suggestions

* Enable filtering by `event:hostname`

* Add tests for filtering by hostnames

* Ensure filter suggestions work for exit pages too

* Allow overriding hostnames with `send_pageview` mix task

* Remove `:window_time_on_page` flag

It seems that we can remove it after all?

* Initialize `experimental_hostname_filter` query parameter

* Rewrite cache store behaviour with regards to session hostnames

* Work around inconsistent session merging

So that `populate_stats` can get closer to actual ingestion

* Improve top stats test

* Make it possible to filter sessions by entry/exit hostnames

* Update pages tests

* Expose `experimental_hostname_filtering` temporarily in the UI

* Untested yet: also apply experimental filtering to sources

* Introduce `hostname_filter` feature flag

* Format

* Test top sources with hostname filter + experimental flag
3 years ago
Adrian Gruntkowski e6d83e946f
Populate new columns in imports and exports (#3969)
* Extend `Imported*` schemas with newly added columns

* Populate newly added `Imported*` fields in GA4 imports

* Extend exports with newly added fields

* Extend CSV importer to ingest new fields

* Fix alias shadowing error

* Add more extensive GA4 import fixtures

* Apply rounding and casting to sampled visits
3 years ago
RobertJoonas bd73fc8266
CH Migration - add more imported metrics and properties (#3949)
* add migration

* add utm_source to imported_sources

* quickfix

* satisfy credo

* Revert "satisfy credo"

This reverts commit bb0b228164.

* Revert "quickfix"

This reverts commit ab6f70c79e.

---------

Co-authored-by: ruslandoga <67764432+ruslandoga@users.noreply.github.com>
3 years ago
Karl-Aksel Puulmann 8e9b4a1840 Breakdowns retain old behavior if getting metric visitors 3 years ago
Karl-Aksel Puulmann 98012c2e31 Simplify code 3 years ago
Karl-Aksel Puulmann 6bb2740e99 Improve naming 3 years ago
Karl-Aksel Puulmann 3d70b4d077 both => sample_percent 3 years ago
ruslandoga 2ba4988c95
specify insert columns in import from s3 (#3968)
Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
3 years ago
Karl-Aksel Puulmann 17a5a5e2c8 TableDecider tests 3 years ago
Karl-Aksel Puulmann cb97cc096c Remove debug code 3 years ago
Karl-Aksel Puulmann fba617c6f4 Handle percentage separately from other metrics 3 years ago
Karl-Aksel Puulmann de1b38ed8b Move TEST_EXPERIMENTAL_REDUCED_JOINS check 3 years ago
Karl-Aksel Puulmann 202d6acbef Custom build name 3 years ago
Karl-Aksel Puulmann 32a1d7c3cd Fix matrix 3 years ago
Karl-Aksel Puulmann a9a404ed09 moduledocs for TableDecider 3 years ago
Karl-Aksel Puulmann 07291b7522 Run tests with experimental_reduced_joins as a separate job
Also refactor which tests are run with postgres:15 to reduce number of jobs
3 years ago
Karl-Aksel Puulmann d3e8965c78 Add support for (plain) breakdowns deciding which table to use 3 years ago
Karl-Aksel Puulmann eed662fa40 events, pageviews are strictly event metrics 3 years ago
Karl-Aksel Puulmann eb8f1e2543 both -> either 3 years ago
Karl-Aksel Puulmann f5b3c7e026 Handle visits, pageviews, events and visitors metrics from other table 3 years ago
Karl-Aksel Puulmann 4ffb25dcaf Dont filter by session properties on events table if querying sessions and joining in events 3 years ago
Karl-Aksel Puulmann 83a6e244ef Make query_events private 3 years ago
Karl-Aksel Puulmann ae4b6504c9 Update test which created sessions/events differently from everyone else 3 years ago
Karl-Aksel Puulmann 7b61eaf047 Handle entry/exit page visit props separately from props stored in events table 3 years ago
Karl-Aksel Puulmann 3cb85d6b8f Fix assignment 3 years ago
Karl-Aksel Puulmann ffb4133cd6 Add some TODOs 3 years ago
Karl-Aksel Puulmann 450c63dd26 Simplify events_join_sessions? 3 years ago
Karl-Aksel Puulmann 2130bc2f3a Make default sessions join more conditional 3 years ago
Karl-Aksel Puulmann 6e415d118a Allow filtering by visit props on event queries if flag is on 3 years ago
Karl-Aksel Puulmann 38880e1401 Remove a hanging TODO 3 years ago
Karl-Aksel Puulmann d2a999b728 Include sample_percent in both tables 3 years ago
Karl-Aksel Puulmann 563da1a382 both -> either 3 years ago
Karl-Aksel Puulmann 960e15e25b WIP new querying system for deciding what tables to query 3 years ago
Karl-Aksel Puulmann 08211661cb Move experimental_session_count? logic to within query object 3 years ago
hq1 1f778e0c11
CH Migration: exit page hostname on sessions_v2 (#3953)
* CH Migration: exit/entry hostnames in sessions_v2

* Leave only exit_page_hostname, we already record hostnames

* Use ClickHouse DDL in favour of ecto so that cluster is included

* Compress with ZSTD(3)
3 years ago
Adrian Gruntkowski 9f27fa303c
Fix dry run mode in `DataMigration.SiteImports` (#3965) 3 years ago
Adrian Gruntkowski 23a3699dd7
Improve import stats toggle and `with_imported` flag computation (#3960)
* Check import presence across all imports and not just the first one

Also, simplify imported data toggle rendering to not explicitly
refer to the earliest import source.

* Change imported stats toggle icon in dashboard

* Test `Imported.get_imports_date_range/1`

* Simplify failed UA/GA import email copy
3 years ago
Adrian Gruntkowski 71fe541359
Implement script for backfilling legacy site import entries and adjusting end dates of site imports (#3954)
* Always select and clear import ID 0 when referring to legacy imports

* Implement script for adding site import entries and adjusting end dates

* Log cases where end date computation is using fallback

* Don't log queries when running the migration to reduce noise
3 years ago
Adrian Gruntkowski 5bf59d1d8a
Implement adjusting imported date range to actual and existing stats (#3943)
* Implement adjusting imported date range to actual and existing stats

* Drop redundant prefix from import list entries

* Make pageview numbers in imports list formatted for readability

* Test and improve date range cropping

* DRY UA and GA4 stats start and end date API calls

* Extend UA/GA import controller tests and improve error handling

* refactor finding longest open range without existing data

* Fix typo in test description

Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>

* Rename `open_ranges` to `free_ranges`

---------

Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
3 years ago
ruslandoga c263df5805
CSV imports (UI) (#3845)
* add basic ui

* remove TODO

* credo

* allow folder upload

* redirect external

* mention folder, use folder icon for file picker

* back to multiple file upload

* mention zip

* escape dots in archive filename
3 years ago
hq1 b31433a7bf
Ensure all the react container attributes are strings (#3948) 3 years ago
hq1 edf70d14b6
Use sessionStorage for "dashboard first launch" banner tracking (#3892)
* Use sessionStorage for offer e-mail report banner tracking

Keeping it within the cookie is problematic, as the banners don't
expire and overflow the cookie with data when enough new sites
are added.

Ref https://github.com/plausible/analytics/issues/3762

* Update changelog

* Extract a component

* Make is_dbip evaluate to quoted boolean
3 years ago
hq1 7523abe93e
Add metrics to ingestion pipeline (#3927)
* Add metrics to ingestion pipeline

* Format

* Format

* Update buckets

* Credo
3 years ago
Karl-Aksel Puulmann 604bf88451
Add github action to validate whether migrations and app change at the same time (#3945) 3 years ago
Karl-Aksel Puulmann 4af7019011
Ignore sessions without entry/exit pages when breaking down entry/exit pages (#3933)
* Ignore sessions without entry/exit pages when breaking down entry/exit pages

* Update stats controller tests to have more realistic test data (pageview followed by event)
3 years ago
hq1 2fae0146a4
Reapply 3918 (#3940)
* Reapply "Pages shield (#3918)"

This reverts commit 33b5c10654.

* Make the FF check work against the site actor
3 years ago
hq1 9989ce6927
Migration for 3918 (#3939)
* Revert "Pages shield (#3918)"

This reverts commit 53f94a9f82.

* Migration: Shield page rules
3 years ago