* add tests for filtering by goal in timeseries and aggregate
* refactor filter parsing
* stop returning custom props in event:goal breakdown
* test breaking down wildcard pageview goals
* extract filter utils
* parse more goal filter options
* add passing tests for new filter types
* do not allow querying session metrics with a goal filter
* remove unused page_match property
* test that non-configured goals are not returned in breakdown
* enforce filtered goals configured
* update changelog
* Allow simple filtering by revenue goals
This does not mean that revenue metrics are supported. If a revenue goal
is filtered by, we treat it like a simple custom event goal in the API.
* use List.wrap
* Disable super-admin checks on small build
* Mute a test writing to stdout
* Move sampling outside of small build
* Convert waiting_first_pageview to heex and stop relying on env vars
* Set site limit unlimited on small build
* Stop relying on app env to get trial expiry
* Remove custom domains - including migration
* Remove is_selfhosted from layout view
* Quota fixup
* Stop relying on app env for self hosted registration
* Stop relying on app env for pass reset success
* Apply on_trial? check only on full build
* Update templates relying on app env
* Adjusts auth controller tests for small build
* Trial fixup
* Fixup
* Stop relying on app env
* Rest of the fsckn owl
* Update typespecs
* Fix dialyzer warning
* Remove unused module
* Credo + format
* GeoIP is not, for full build
* Use `small_build?()` where applicable
* Implement bypassing FirstLaunchPlug without insertions
* Get Marko's patch de58a18a85
* Test is-dbip=false presence
* Fix typespec
* Remove future hardcodes
* Handle `nil` from `Plausible.Geo.database_type()`
* Remove XXX marker
* Use one typespec for two clauses
* Introduce `MIX_ENV=small_dev`
* Revert "Use one typespec for two clauses"
This reverts commit 8d8cd21764.
* Include ApiKey functions in Auth context
* Make feature notice work without %Site{}
Previously the extra feature notice required a %Site{} in order to check
the owner plan. However, not every feature is scoped by site, for
example the Stats API. For features like this, a %User{} is required,
and not a %Site{}.
This commit replaces the `:site` param with `:billable_user`, which is
common to both site and user-scoped features.
* Add stats_api to the list of extra features
* Limit API Key creation based on user plan
* Make membership creation and role updates more explicit in terms of changesets
* Extract invitation accept flow logic and refactor it slightly
* Improve acceptation logic
* Update moduledoc
* Improve SiteLocker API and add typespecs
* Stop naming function not returning a boolean like a predicate
* Refactor rest of invitation actions and safeguard against rogue requests
* Update code docs slightly
* Extend `Billing.check_needs_update/1` tests
* Parametrize selfhost flag and toggle SiteLocker logic on it
* Add tests for newly extracted services
* Add test case and a fix for locking site on grace period ended
* Make invitation controller tests async as there's no more env patching
* Add test cases for self-invites and fix one bug
* Add and refactor tests for rejecting and removing invitations
* Prevent issuing ownership transfer to existing owner
* Improve name of the test
* Improve `Billing.check_needs_to_upgrade/1` return value
* Improve `Billing.SiteLocker.update_sites_for/1` and its tests
* Fix typos
Co-authored-by: hq1 <hq@mtod.org>
* Make invitation removal and rejection resilient to races
---------
Co-authored-by: hq1 <hq@mtod.org>
* Filter out empty entries when listing stats for UTM props
* Update test fixtures removing noref entries in UTM CSV stat exports
* Update external API tests to account for lack of noref records for UTM stats
* Filter out entries with empty UTM props from imported GA stats
* Remove unreachable GA utm_source dim clause in imported stats logic
* default to source_query.include_imported before true
This fixes a bug where a Stats API aggregate query was trying
to query current period *without imported data* but previous
period *with imported data*.
* remove __internal_visits at a better time
As soon as we have made the db query, we don't need it anymore
* disallow events metric with imported data
* update changelog
* keep default in a better place
* remove unused option
* update test description
Co-authored-by: Vini Brasil <vini@hey.com>
---------
Co-authored-by: Vini Brasil <vini@hey.com>
* add entry page filter condition to bounce rate
* Fix select_merge with dynamic
* fix tests and add one test
* generalize page_filter_condition function
* use dynamic_filter_condition for session filters too
* disable views_per_visit with a page filter
* update changelog
* disable credo for complex function
---------
Co-authored-by: Uku Taht <uku.taht@gmail.com>
* Add PropFilterModal (only UI)
* small variable refactor
* allow selecting prop value filter type
* allow selecting only one prop_key
* allow selecting many prop_values only when prop_key selected
* handle submitting filter
* get applied filters from query + remove option
* change prop filter label format
* support member and not_member filter types for pageview props
* show (none) value in filter suggestions
* refactor zip_results/4 and remove unused code
* fix displaying (none) values in goals section prop breakdown
* remove unnecessary functionality
* fix bug: returning prop names for goal :member filter
* fix bug: submitting regular filter modal with Enter key
* bugfix: disallow opening prop filter modal when feature flag disabled
* mix format
* break selected combobox values into multiple lines
* fix useEffect behavior for focusing on prop_key field
* support submitting prop filter with Enter key
* refactor getFormState in PropFilterModal
* separate fetchPropKey and fetchPropValue functions
* Allow querying props for pageview goals
* Make the internal props API only return a list of props (not map)
* Separate function for fetching all props in Stats API goal breakdown (this returns a map as before)
* ditch state for keeping search bar visible
* group by event_name in db query
* Migration (PR: https://github.com/plausible/analytics/pull/2802)
* Implement Site.Domain interface allowing change and expiry
* Fixup seeds so they work with V2_MIGRATION_DONE=1
* Update Sites.Cache so it's capable of multi-keyed lookups
* Implement worker handling domain change expiration
* Implement domain change UI
* Implement transition period for public APIs
* Exclude v2 tests in primary test run
* Update lib/plausible_web/controllers/site_controller.ex
Co-authored-by: Vini Brasil <vini@hey.com>
* Update lib/plausible_web/controllers/site_controller.ex
Co-authored-by: Vini Brasil <vini@hey.com>
* Update moduledoc
* Update changelog
* Remove remnant from previous implementation attempt
* !fixup
* !fixup
* Implement domain change via Sites API
cc @ukutaht
* Update CHANGELOG
* Credo
* !fixup commit missing tests
* Allow continuous domain change within the same site
---------
Co-authored-by: Vini Brasil <vini@hey.com>
* Remove ClickhouseSetup module
This has been an implicit point of contact to many
tests. From now on the goal is for each test to maintain
its own, isolated setup so that no accidental clashes
and implicit assumptions are relied upon.
* Implement v2 schema check
An environment variable V2_MIGRATION_DONE acts like
a feature flag, switching plausible from using old events/sessions
schemas to v2 schemas introduced by NumericIDs migration.
* Run both test suites sequentially
While the code for v1 and v2 schemas must be kept still,
we will from now on run tests against both code paths.
Secondary test run will set V2_MIGRATION_DONE=1 variable,
thus making all `Plausible.v2?()` checks return `true'.
* Remove unused function
This is a remnant from the short period when
we would check for existing events before allowing
creating a new site.
* Update test setups/factories with v2 migration check
* Make GateKeeper return site id along with :allow
* Make Billing module check for v2 schema
* Make ingestion aware of v2 schema
* Disable site transfers for when v2 is live
In a separate changeset we will implement simplified
site transfer for when v2 migration is complete.
The new transfer will only rename the site domain in postgres
and keep track of the original site prior to the transfer
so we keep an ingestion grace period until the customers
redeploy their scripting.
* Make Stats base queries aware of v2 schema switch
* Update breakdown with v2 conditionals
* Update pageview local start with v2 check
* Update current visitoris with v2 check
* Update stats controller with v2 checks
* Update external controller with v2 checks
* Update remaining tests with proper fixtures
* Rewrite redundant assignment
* Remove unused alias
* Mute credo, this is not the right time
* Add test_helper prompt
* Fetch priv dir so it works with a release
* Fetch distinct partitions only
* Don't limit inspect output for partitions
* Ensure SQL is printed to IO
* Remove redundant domain fixture
* globally rename 'pages_per_visit' to 'views_per_visit'
* change the order of top stats
* rename 'Visits' to 'Total visits' in the UI
* add views_per_visit to UI
* put the new metric under a feature flag
* add new metric to CSV export under feature flag
* mix format
* use only one feature flag
* refactor metric validation
* link to the correct docs section
* add the new metric to aggregate API
* explicitly remove __internal_visits
The overall metric selection is well defined by
`Plausible.Stats.Timeseries.empty_row/2`. The only metric that needs
to be removed from the timeseries response is __internal_visits.
This commit also moves the `remove_internal_visits_metric` function to a
new Util module to be used by both breakdown and timeseries.
* add the new metric to timeseries API
* mix format
* add moduledoc to keep credo happy
* convert pages_per_visit to string straight away
* do rounding in db query
* query # of visits with sum(sign) instead
* stop converting pages_per_visit to string
* fix subquery for sessions in base_event_query/2
As the 'sessions' table is using the CollapsingMergeTree engine, we have
to select session_id's distinctively. Otherwise we will get multiple rows
(with sign -1 and 1) as long as the background merge hasn't happened.
* update changelog
* use GROUP BY instead of SELECT DISTINCT
* remove comma
* Fix breakdown API pagination when using event metrics
This commit fixes a bug where the subsequent breakdown API pages had
the same items as the first page. The fix sorts the underlying
ClickHouse query by timestamp, keeping the same order between requests,
as we use OFFSET/LIMIT pagination.
* Fix repeated results assertion
* Add different ORDER BY to each breakdown property
* Return empty list when breaking down by event:page without events
This commit fixes a bug with pagination where breaking down by event:page
would always return results despite pagination.
Closes#2255
* Update CHANGELOG.md
* Make TestUtils module available in all tests
* Add macros patching the application env in tests
Unfortunately a lot of existing functionality relies on
certain application env setup. This isn't ideal because
the app config is a shared state that prevents us from
running the tests in parallel.
Those macros encapsulate setting up new env for test purposes
and make sure the changes are reverted when the test finishes.
* Allow passing request opts to HTTPClient.post/4
We need this to swap custom request building in
Google Analytics import.
* Unify errors when listing sites
* React: propagate backend error messages if available
* React: catch API errors in Search Terms component
* Propagate google API errors on referrer drilldown
* Handle verified properties errors in SC settings
* Add missing tests for SC settings controller
* Unify errors for fetching search analytics queries (list stats)
* Unify errors refreshing Google Auth Token
* Test fetch_stats/3 errors and replace Double with Mox
* Fixup makrup
* s/class/className
* Simplify Search Terms display in case of errors
* Fix warnings
* add separate module for filter parsing
* add tests for filter parser
* allow escaping pipe character in filter value
* add documentation and doctests
* do not remove escape chars from wildcard values
* changelog update
* change the parse_filters/1 function argument
* Render 404 when shared link cannot be found
* Add documentation for StatsController and shared link rendering
* Refactor shared_link/2 for more clarity
* Add changelog entry
* Use mermaid graph for sequence diagram
* Use more accurate return value in sequence diagram
* Refactor Ecto query to be more idiomatic
* Remove order dependence in test
* Restore backwards compatibility for older shared links
* Add changelog entry
* can now query events metric when breaking down on a visit prop
* updated changelog
* fixed formatting
* fixed a bug - querying for bounce rate does not return visits anymore
* removed the changelog update
* Adds pagination for custom props, fixes errant (none) being returned in the middle of a dataset
* Formatting
* Fixes errant (none) value correctly
* Changelog
* Adds tests
* Show hourly data for a day
* Fix aggregations with long periods and comaprisons enabled
* Return Direct / None instead of empty string when missing source
* Fix/Align breakdown query with current queries in the UI
* Show correct number of days for 7d and 30d
* Wrap results of aggregate call in `results`
* Allow breakdown by event:name
* Ensure event:name filter works in all contexts
* WIP
* Add support for custom event props