Commit Graph

2571 Commits (dea210469e121b3d2da4fa3ca101fbb71f5423e0)
 

Author SHA1 Message Date
Adam Rutkowski dea210469e wip 3 years ago
Adam Rutkowski cf59c1e572 wip 3 years ago
Adam Rutkowski a7ceec21ec wip 3 years ago
Adam Rutkowski 515a9f680b wip 3 years ago
Adam Rutkowski 013177f25b wip 3 years ago
Adam Rutkowski 55c5e4d09b CH Migration: exit/entry hostnames in sessions_v2 3 years ago
Adrian Gruntkowski 5bf59d1d8a
Implement adjusting imported date range to actual and existing stats (#3943)
* Implement adjusting imported date range to actual and existing stats

* Drop redundant prefix from import list entries

* Make pageview numbers in imports list formatted for readability

* Test and improve date range cropping

* DRY UA and GA4 stats start and end date API calls

* Extend UA/GA import controller tests and improve error handling

* refactor finding longest open range without existing data

* Fix typo in test description

Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>

* Rename `open_ranges` to `free_ranges`

---------

Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
3 years ago
ruslandoga c263df5805
CSV imports (UI) (#3845)
* add basic ui

* remove TODO

* credo

* allow folder upload

* redirect external

* mention folder, use folder icon for file picker

* back to multiple file upload

* mention zip

* escape dots in archive filename
3 years ago
hq1 b31433a7bf
Ensure all the react container attributes are strings (#3948) 3 years ago
hq1 edf70d14b6
Use sessionStorage for "dashboard first launch" banner tracking (#3892)
* Use sessionStorage for offer e-mail report banner tracking

Keeping it within the cookie is problematic, as the banners don't
expire and overflow the cookie with data when enough new sites
are added.

Ref https://github.com/plausible/analytics/issues/3762

* Update changelog

* Extract a component

* Make is_dbip evaluate to quoted boolean
3 years ago
hq1 7523abe93e
Add metrics to ingestion pipeline (#3927)
* Add metrics to ingestion pipeline

* Format

* Format

* Update buckets

* Credo
3 years ago
Karl-Aksel Puulmann 604bf88451
Add github action to validate whether migrations and app change at the same time (#3945) 3 years ago
Karl-Aksel Puulmann 4af7019011
Ignore sessions without entry/exit pages when breaking down entry/exit pages (#3933)
* Ignore sessions without entry/exit pages when breaking down entry/exit pages

* Update stats controller tests to have more realistic test data (pageview followed by event)
3 years ago
hq1 2fae0146a4
Reapply 3918 (#3940)
* Reapply "Pages shield (#3918)"

This reverts commit 33b5c10654.

* Make the FF check work against the site actor
3 years ago
hq1 9989ce6927
Migration for 3918 (#3939)
* Revert "Pages shield (#3918)"

This reverts commit 53f94a9f82.

* Migration: Shield page rules
3 years ago
hq1 53f94a9f82
Pages shield (#3918)
* Migration: Shield page rules

* Add Ecto schema for Page Rules

* Add Page Rule cache

* Fix typo

* BTW: Use already imported function

* Extend Shields context interface + split existing tests

* Ingestion: filter matching patches + refactor shield actions

* Add LV section for adding Page Rules

* Validate max page path length

* Put Pages Shield behind a feature flag

* Update CHANGELOG

* Update docs link anchor

As per https://github.com/plausible/docs/pull/477

* Update lib/plausible/shields.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

* Update lib/plausible_web/live/shields/page_rules.ex

Co-authored-by: ruslandoga <doga.ruslan@gmail.com>

* Update lib/plausible_web/live/shields/page_rules.ex

Co-authored-by: ruslandoga <doga.ruslan@gmail.com>

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
Co-authored-by: ruslandoga <doga.ruslan@gmail.com>
3 years ago
Adrian Gruntkowski ba5b80a8c0
Add label to site imports and populate it (#3914) 3 years ago
Uku Taht 1d017e86a1
Fix escaping of source filters (#3930)
* Fix escaping of source filters

* CHANGELOG

* Fix typo

Co-authored-by: Karl-Aksel Puulmann <macobo@users.noreply.github.com>

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
Co-authored-by: Karl-Aksel Puulmann <macobo@users.noreply.github.com>
3 years ago
Adrian Gruntkowski 52c226c428
Add `label` column to `site_imports` schema (#3926) 3 years ago
RobertJoonas fb61f0b425
Capitalize Total Conversions in graph tooltip (#3934) 3 years ago
RobertJoonas d6e1e8bebd
Put total conversions on the graph + goal-filtered CSV export improvements (#3929)
* Add validation for the events metric in main_graph

* Test the already existing events metric support in main-graph

* Put total conversions on the graph

* extract main_graph_csv function (refactor only)

* add total_conversions and conversion_rate to goal-filtered visitors.csv

* update changelog
3 years ago
Uku Taht 561dcd821e
Mask pathanme in filter menu event (#3932) 3 years ago
Uku Taht fd879eeb16
Store referrers from android apps (#3715)
* Store referrers from android apps

* Add test for unknown referrer protocol

* Store android referrer protocol
3 years ago
Uku Taht 8992c8ee07
Add tracking to filter button (#3928) 3 years ago
RobertJoonas c32779a3e5
Timeseries for conversion rate (#3919)
* add conversion rate to Stats API timeseries

* make sure CR can be queried as the only metric

* add a test asserting zeros are returned

* add tests for filtering by other properties at the same time

* Remove unnecessary validation of params

1. It doesn't make to validate `interval` (and its granularity) in all
   endpoints. It's only relevant for the main graph.

2. The plug (renamed to `date_validation_plug`) already makes sure that
   the dates are validated. No need to call the same function again in
   Top Stats and Funnel endpoints.

* add metric validation to main graph

* Add tests for main graph API

* put conversion rate on the graph

* update changelog

* Add revenue metrics into metrics.ex

* make fn private

* avoid setting graph metric to visitors in goal-filtered view
3 years ago
Adrian Gruntkowski d6e81670e4
Unify UA and GA4 import flow into one (#3888)
* Unify GA4 and UA import flow into one

* Clean up property and view data retrieval via Google HTTP APIs

* Turn `Map.get` into `Map.fetch!` in API response processing code

* Bump list account summaries page size limit to max of 200

* Show only views in legacy flow and fix legacy redirect after import start

* Move google analytics import actions tests to a separate module

* Extend Google Analytics controller tests

* DRY up `property?` predicate (h/t @RobertJoonas)
3 years ago
ruslandoga 5f9465614b
Include domain and dates in zip archive filename (#3921)
* include domain and dates in zip archive filename

* adapt to comments
3 years ago
Karl-Aksel Puulmann 32ab138301
Fix issue with name clash (#3925)
Unexpectedly, table.name caused a name clash after CR refactor, so using a unique name
for the output column

Sentry issue: https://sentry.plausible.io/organizations/sentry/issues/5612
3 years ago
Karl-Aksel Puulmann c219652dae
Re-apply Move conversion_rate logic from elixir to clickhouse (#3924)
* Revert "Revert "Move conversion_rate logic from elixir to clickhouse (#3887)"…"

This reverts commit 253fb5d67d.

* Fix issue with missing columns

The issue came from refactoring event:goal UNION ALL logic and trying to move
name select from first to last. If any other tables were joined, the incorrect
item would be used as an array index, causing this issue.

Added a relevant test.
3 years ago
Karl-Aksel Puulmann 253fb5d67d
Revert "Move conversion_rate logic from elixir to clickhouse (#3887)" (#3923)
This reverts commit 1909743b90.
3 years ago
Karl-Aksel Puulmann 1909743b90
Move conversion_rate logic from elixir to clickhouse (#3887)
* Separate out query building from pagination/execution logic.

* Refactor pageview_goals breakdown query, removing index column from results

* Remove zip_columns logic

* Use common pagination util

* Do everything in a single query for breakdowns for goals

* Order in DB

* Make sure column order is identical

* Calculate CR within the goal breakdown query

* Calculate CR for property breakdowns

* WIP: Calculate group CR

* CR with order_by

* Compatibility fix

* Import Ecto.Query and cleanup

* handle total_visitors the same way as add_percentage

* Handle conversion_rate in aggregate.ex

* Solve rebase fail

* Simplify maybe_add_group_conversion_rate

* Add conversion_rate defaults to 0 test

* Add test for conversion_rate should not be calculated with imported data (failing here and on master)

* Dont include imported data when breakdown by prop or goal

* Remove revenue_nils
3 years ago
ruslandoga 64f35bf8f6
make robots.txt restrictive by default (#3905)
* make robots.txt restrictive by default

* add note about cloud robots.txt
3 years ago
ruslandoga 279e89c693
CSV imports (no UI) (#3895)
* encode/decode date range in filenames

* Update lib/plausible/imported/csv_importer.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

* Update lib/plausible/imported/csv_importer.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

* drop unused functions

* send failure email if there is no data to export

* use PlausibleWeb.Email.mailer_email_from()

* ensure we get dates from minmax date query

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
3 years ago
ruslandoga 4242b52be4
Allow importing extra config (#3906)
* allow importing extra config

* changelog

* fix typo

* add test
3 years ago
ruslandoga dfcc8d794a
Add space after logo on the index page (#3825) 3 years ago
Karl-Aksel Puulmann 02d2256483
Set exit_page only on pageviews (#3870)
* Set exit_page only on pageviews

* Update tests

* Update entry_page on first pageview

* Update CHANGELOG.md
3 years ago
ruslandoga 07b714a143
Update Sentry (#3843)
* update Sentry

* Sentry.HTTPClient.child_spec is now optional

* Sentry.EventFilter is deprecated

* update sentry to 10.2.0

* fix dialyzer warnings
3 years ago
ruslandoga c08aa29ad8
Remove DISABLE_AUTH deprecation warning (#3904)
* remove DISABLE_AUTH deprecation warning

* changelog
3 years ago
ruslandoga 0a0c7ce165
Remove init-admin.sh (#3903)
* remove init-admin.sh

* changelog
3 years ago
ruslandoga 8c0e94f72c
Fix manual embed width (#3910)
* fix manual embed width

* changelog
3 years ago
ruslandoga 4ae00540dc
Split Elixir CI jobs (#3898)
* split elixir tests

* more understandable matrix

* add unused deps in mix.lock check

* get all deps for static checks
3 years ago
ruslandoga 5e74b1cf74
CSV exports (UI) (#3875)
* ui

* fix redirect link

* improve make minio

* use implicit button form for csv export

* add exports_bucket helper

* read S3_EXPORTS_BUCKET

* supply s3_bucket in export_csv job args

* make plausible_minio use unprotected port

* move s3_csv_export queue to base queues

* Update lib/plausible_web/controllers/site_controller.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
3 years ago
Karl-Aksel Puulmann d3586a81e3
UX improvement: don't autofocus on Custom properties on dashboard (#3872)
* UX improvement: don't autofocus on Custom properties on dashboard

Currently when you visit the dashboard with a certain configuration,
it will automatically scroll to the bottom, which is annoying.

To reproduce:
1. localStorage.clear() in console
2. Refresh, open properties at the very bottom
3. Refresh again

Expected behavior:
1. Dashboard shows

Actual behavior:
1. Dashboard scrolls to the bottom

* Changelog entry
3 years ago
hq1 59afa20955
Reapply #3878 + bugfix hit rate tracking (#3891)
* Reapply "Replace caching engine (#3878)" (#3883)

This reverts commit c5881cdc6d.

* Ensure hit rate is tracked on `get_or_store`

* Remove :wx and :observer

* Remove unused deps

* Use `:set` table type
3 years ago
RobertJoonas e8f3946dde
Fix division by zero in imported queries (#3890)
* prevent division by 0 in merge_imported queries

* Revert "fix bounce_rate change bug (#3886)"

This reverts commit 6eef32a8ff.

After 02aa0b2, we can keep on assuming that bounce rate is always numeric.
3 years ago
Adrian Gruntkowski 4d7d88cfec
Implement basics of GA4 import (#3851)
* Implement LV date input using flatpickr

* Implement basics of GA4 import (very dirty WIP)

* Split Google HTTP API into UA and GA4 specific parts

* Add a quick way to record GA4 API responses

* Add first GA4 import fixtures with GA4 Data API responses

* Extract GA4 and UA specific logic form Google API

* Extract UA and GA4 specific actions to distinct controllers

* Add integration test for GA4 importer

* Update GA4 fixtures

* Test GA4 API

* Add debug logging and fix paginating through API results in in GA4 import

* Revert "Implement LV date input using flatpickr"

This reverts commit c696f8ee39d5702f27015c09a4f079ca124cc7bb.

* Fix note
3 years ago
ruslandoga f2350b5165
Add `/tmp/` to .gitignore and simplify s3 cleanups in tests (#3889) 3 years ago
ruslandoga 5a3072ca21
CSV exports (no UI) (#3836)
* csv exports

* use ex_unit's tmp_dir
3 years ago
RobertJoonas 6eef32a8ff
fix bounce_rate change bug (#3886) 3 years ago
RobertJoonas 7641c66a2b
Stats api time on page (#3858)
* add metric validation + support in aggregate

* add a test ensuring comparison works

* disallow time_on_page with a goal filter

* Return time_on_page as `nil` from aggregate API

In case time_on_page cannot be calculated, we'll return it as `nil` from
the Stats API.

This is to make the behaviour consistent between breakdown and aggregate
endpoints. As for the UI, we'll still continue to report time_on_page as
0 - not changing any UI behaviour as discussed with Marko.

* add tests for time_on_page in event:page breakdown

* update changelog

* invalidate time_on_page with event:name filter

* add the ability to only query time_on_page in page breakdown

We'll need the visitors metric to get the list of pages to calculate the
time_on_page for.
3 years ago