Commit Graph

280 Commits (f3423aefec68845895e5cd2e09bb93bda5bd73de)

Author SHA1 Message Date
hq1 518cdb3307
Shield: Country Rules (#3828)
* Migration: add country rules

* Add CountryRule schema

* Implement CountryRule cache

* Add country rules context interface

* Start country rules cache

* Lookup country rules on ingestion

* Remove :shields feature flag from test helpers

* Add nested sidebar menu for Shields

* Fix typo

* IP Rules: hide description on mobile view

* Prepare SiteController to handle multiple shield types

* Seed some country shield

* Implement LV for country rules

* Remove "YOU" indicator from country rules

* Fix small build

* Format

* Update typespecs

* Make docs link point at /countries

* Fix flash on top of modal for Safari

* Build the rule struct with site_id provided up-front

* Clarify why we're messaging the ComboBox component

* Re-open combobox suggestions after pressing Escape

* Update changelog

* Fix font size in country table cells

* Pass `added_by` via rule add options

* Display site's timezone timestamps in rule tooltips

* Display formatted timestamps in site's timezone

And simplify+test Timezone module; an input timestamp converted
to UTC can never be ambiguous.

* Remove no-op atom

* Display the maximum number of rules when reached

* Improve readability of remove button tests

* Credo

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
3 years ago
RobertJoonas dd428430f5
Query imported data for views_per_visit (#3830)
* query imported data for views_per_visit

* changelog update
3 years ago
RobertJoonas 52f584efa9
Group `os_version` by `os` (#3806)
* improve test

* add os to os_version breakdown

...and add operating_system_versions.csv to the CSV export

* fix conversion rate for os_version breakdown

* update changelog

* fix existing CSV tests

* use case instead of cond
3 years ago
RobertJoonas d74b1d5e60
Reapply conversion rate into Stats API + bugfixes (#3805)
* Revert "Revert api conversion rate (#3789)"

This reverts commit 8e8790dd30.

* fix browser_version CR breakdown bug

* changelog bugfix

* inspect data structures before sending to sentry
3 years ago
RobertJoonas 8e8790dd30
Revert api conversion rate (#3789)
* Revert "Unify percentage change for CR and bounce_rate (#3781)"

This reverts commit a6b1a6ebc7.

* Revert "Bring Stats API up to speed: Add `conversion_rate` to Aggregate and Breakdown (#3739)"

This reverts commit 672d682e95.
3 years ago
RobertJoonas a6b1a6ebc7
Unify percentage change for CR and bounce_rate (#3781)
* Fix conversion rate change calculation

The change in conversion rate should be calculated similar to bounce rate.
For example, an increase of 25% -> 50% should not be a 100% change, but
a 25% change instead.

* Use the same comparison function in Stats API and dashboard API

This commit fixes a bug where the percentage change reported by the Stats
API is different from the one returned by the internal dashboard API.

* changelog update
3 years ago
hq1 926de4dd10
Experimental session count (#3786)
* WIP

* Allow `experimetnal_session_count` request serialization

* Extend `Plausible.Stats.Query` with `experimental_session_count` flag

* Add `FunWithFlags` actor implementation for `Site`

* Change the way sessions are retrieved

* Remove redundant test

* Format

* Update the test

---------

Co-authored-by: Uku Taht <uku.taht@gmail.com>
3 years ago
RobertJoonas 672d682e95
Bring Stats API up to speed: Add `conversion_rate` to Aggregate and Breakdown (#3739)
* disable event metric with include_imported in every case

* add missing test for metric validation

* refactor metric validation functions

* implement conversion_rate metric validation

* move calculate_cr function into Stats.Util

* Refactor: Move aggregate CR logic into Stats.aggregate

* define atoms to exist

* Ensure that CR does not depend on visitors being queried

If 'visitors' are already queried, we'll use that value. Otherwise we'll
need to make another query to fetch it.

* confirm Stats API aggregate supports CR (tests only)

* small refactor

This is the only 'event_property' left after pattern matching on all
others in the function clauses defined above.

* Make it possible to optionally query conversion_rate

...in breakdown queries (excluding goal and custom prop breakdown)

* A little refactor asking for revenue metrics

1. The `@revenue_metrics` module attribute is an empty list on full build
   anyway
2. We don't need to query for revenue metrics if there are no revenue goals
   returned in the given query (even if revenue goals exist in site.goals)
3. Revenue metrics are already dropped in prop breakdown without a goal
   filter via (get_revenue_tracking_currency/3)

* Make it possible to optionally query conversion_rate (continuation)

... also from a custom prop and goal breakdown

* Frontend adjustments to the Locations report

* Display conversion rate in Regions and Cities (ListReport view)
* Display total conversions, conversions (visitors), and CR in the
  "Details" modals of Countries, Regions, and Cities
* Move the percentage into a separate column in the Countries details table

* confirm Stats API breakdown supports conversion_rate (tests only)

* small refactor: extract maybe_add_time_on_page function

* Make it possible to query cr alone

... (without the visitors metric). Already supported in aggregate, this
commit only implements it for the breakdown API.

* Reuse Stats.Util helper functions from b02db88 for aggregate API

We can follow the same logic as with breakdown for manually adding
`visitors` into the metrics list and taking it out of the response
later on.

That way we don't have to make another query, e.g. in a case where
only pageviews and conversion rate is queried. Also keeps things
consistent.

* changelog update

* fix test after resolving merge conflict

* Use explicit string->atom mapping instead of casting

* alias Util module instead of importing it

* use Enum.empty instead of Enum.any

* improve readability

* rename special_metrics to computed_metrics and explain with a comment

* rename visitors_without_event_filters to total_visitors

* keep a single function for removing unwanted metrics

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
3 years ago
Adrian Gruntkowski f8b4d5066a
Add multiple imports per site (#3724)
* Clean up references to no longer active `google_analytics_imports` Oban queue

* Stub CSV importer

* Add SiteImport schema

* Rename `Plausible.Imported` module file to match module name

* Add `import_id` column to `Imported.*` CH schemas

* Implement Importer behavior and manage imports state using new entities

* Implement importer callbacks and maintain site.imported_data for UA

* Keep imports in sync when forgetting all imports

* Scope imported data queries to completed import IDs

* Mark newly imported data with respective import ID

* Clean up Importer implementation a bit

* Test querying legacy and new imported data

* Send Oban notifications on import worker failure too

* Fix checking for forgettable imports and remove redundant function

* Fix UA integration test

* Change site import source to atom enum and add source label

* Add typespecs and reduce repetition in `Plausible.Imported`

* Improve documentation and typespecs

* Add test for purging particular import

* Switch email notification templates depending on import source

* Document running import synchronously

* Fix UA importer args parsing and ensure it's covered by tests

* Clear `site.stats_start_date` on complete import to force recalculation

* Test Oban notifications (h/t @ruslandoga)

* Purge stats on import failure right away to reduce a chance of leaving debris behind

* Fix typos

Co-authored-by: hq1 <hq@mtod.org>

* Fix another typo

* Refactor fetching earliest import and earliest stats start date

* Use `Date.after?` instead of `Timex.after?`

* Cache import data in site virtual fields and limit queried imports to 5

* Ensure always current `stats_start_date` is used

* Work around broken typespec in Timex

* Make `SiteController.forget_imported` action idempotent

* Discard irrecoverably failed import tasks

* Use macros for site import statuses

There's also a fix ensuring only complete imports are considered
where relevant - couldn't isolate it as it was in a common hunk

* Use `import_id` as worker job uniqueness criterion

* Do not load imported stats data in plugins API context

---------

Co-authored-by: hq1 <hq@mtod.org>
3 years ago
Karl-Aksel Puulmann f3509f2a17
Refactor spike detection top sources query (#3770)
* ORDER BY referrer_source for spikes job

This is more consistent with the rest of the queries

* Refactor top_sources -> top_sources_for_spike

* Remove more dead code

* Remove unused arguments

* Remove unused select arguments

* Add a test to top_sources_for_spike
3 years ago
Karl-Aksel Puulmann d1fe184cb7
Remove ORDER BY min(session.start) (#3771)
This adds a needless performance overhead. Note that all queries already
sort by the _name_ of the breakdown value besides the session start.
3 years ago
Karl-Aksel Puulmann 1cb7982cd9
Filtering by multiple custom properties (#3719)
* WIP: PropFilterRow

* Get multi-behavior working

* Render multiple prop filters in one

* Modal reads from query string correctly

* Backend support for multiple custom property filters

* Add backend tests for multiple custom property filters

* Disable already selected options in property keys

We can't allow choosing the same property multiple times without changing the request
params, which we decided against

* Allow choosing any property under Behaviors > Custom props even if custom prop filter applied

This was a limitation (I believe) introduced by using ARRAY JOINs to query custom properties

* CHANGELOG.md

* Solve credo warning about too deep nesting

* Update assets/js/dashboard/stats/modals/prop-filter-modal.js

Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>

* Refactor internal function for clarity

* Add another step -> Add another

* Solve 500 error

* Separate boxes per property filter

* Retain other filters in props table

* removeFilter behavior for props

* matches_member support for custom props

* filter_suggestions for prop keys should account for prop filter

* find over filter

* refactor appliedFilters

* FILTER_TYPES => FILTER_OPERATIONS

* Make add another link not wrap the whole page

* Unique keys

---------

Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
3 years ago
Adrian Gruntkowski 3738cd9578
Parse referrer filter value when passed separately (#3742)
* Parse referrer filter value when passed separately

* Remove unnecessary test setup
3 years ago
hq1 931161f693
Generalize Cache/Warmers - extract only specifics to `Site.Cache` (#3716)
* Simplify caches config

The intervals were proven to never change really,
and previous shape made them not very discoverable either.
We'll look into a more declarative setup.

* Aim for more declarative supervision tree setup

* Generalize `Plausible.Cache`

This makes the `Site.Cache` module implement the specifics,
leaving the common bits reusable for upcoming cache processes.

* Generalize `Plausible.Cache` warmers

* Fix typos
3 years ago
RobertJoonas 04b9067591
Add `event:goal` property to Stats API (#3718)
* add tests for filtering by goal in timeseries and aggregate

* refactor filter parsing

* stop returning custom props in event:goal breakdown

* test breaking down wildcard pageview goals

* extract filter utils

* parse more goal filter options

* add passing tests for new filter types

* do not allow querying session metrics with a goal filter

* remove unused page_match property

* test that non-configured goals are not returned in breakdown

* enforce filtered goals configured

* update changelog

* Allow simple filtering by revenue goals

This does not mean that revenue metrics are supported. If a revenue goal
is filtered by, we treat it like a simple custom event goal in the API.

* use List.wrap
3 years ago
hq1 0fa7f3c1e1
Make subsequent query boundaries idempotent (#3723)
* Make subsequent query boundaries idempotent

This should prevent time-sensitive race conditions
on subsequent `Plausible.Stats.*` calls for the same query.
Not sure if there are more places where now() is
dynamic relative to query life cycle, but it's a start.

* Format

* Update lib/plausible/stats/query.ex

Co-authored-by: Wojtek Mach <wojtekmach@users.noreply.github.com>

* Match on `Query.now` in a test

---------

Co-authored-by: Wojtek Mach <wojtekmach@users.noreply.github.com>
3 years ago
Karl-Aksel Puulmann 11c5d3b251
Remove ARRAY JOINs (#3708)
* Use array indexing instead of left joins

This has multiple benefits:
- Allows us to more easily add support for multiple custom property
  filters
- Performance: Increased performance of some queries by ~20%

Work left to do:
- Check tests coverage
- Create macros for repeated fragments
- Quote and refer to right tables in fragments/macros

* Use macros to clean up code

* Use existing atoms to avoid memory leaks
3 years ago
RobertJoonas 041b0ab779
trace aggregate and timeseries queries too (#3712) 3 years ago
RobertJoonas 6ca4eaf5e2
Improve query tracing (#3704)
* Enable breakdown property tracing for event:goal

This is currently overwritten by event:name. The queries for event:name
will still be tracked if queried specifically.

* Add filters tracing to Query.trace/1

* make sure breakdown metrics are sorted and joined

* check opts skip tracing directly without passing it
3 years ago
Karl-Aksel Puulmann 1b95433d6e
Support `contains`/`matches` for custom properties (#3687)
* Allow `matches` operator to work in BE for custom props

Note: No FE support yet, needs further testing

* feat: allow choosing `contains` for property filters in the UI

* no autocomplete on prop values if `contains` for consistency

* CHANGELOG.md

* Fix: Handle (none) property in property breakdowns when using matching

When matching we should always exclude (none)
3 years ago
Uku Taht a5e0619053
Escape special regex characters (#3634)
* Escape special regex characters

* Add changelog entry
3 years ago
RobertJoonas 7536f0285c
Fix weekly interval bug (#3686)
* Fix a bug

This commit fixes a bug where the timeseries weekly interval query for a
month returned 0 visitors for the first week.

This happened because the start of the month defining the first week was
missing the UTC -> site.timezone conversion.

* Update test/plausible_web/controllers/api/stats_controller/main_graph_test.exs

Remove dot from test description

Co-authored-by: Karl-Aksel Puulmann <macobo@users.noreply.github.com>

---------

Co-authored-by: Karl-Aksel Puulmann <macobo@users.noreply.github.com>
3 years ago
RobertJoonas 403f559b35
Limit Custom Properties access in Stats API (#3670)
* add new function and doc for allowed_props

* limit props access in Stats API queries

* use dot syntax instead
3 years ago
hq1 5c8f39ac4c
Prevent crashes on ambiguity when casting timezones (#3663)
* Fall back to UTC when timezone gap found

* bugfix 1

* bugfix 2

* Apply safe tz conversion to all cases where it's done on a ts other than now

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
3 years ago
RobertJoonas 155a7a56a7
Fix goal filter inconsistency (#3658)
* remove unused code

* add e.name == "pageview" condition to pageview goals

This fixes the weird behavior where filtering by a pageview goal would
also return custom events and vice versa.

* update changelog
3 years ago
hq1 8be9397199
Skip breakdown of imported pages with pageviews=0 (#3655)
* Skip breakdown of imported pages with pageviews=0

* Fixup

* Prove good impoted records are merged
3 years ago
RobertJoonas d2270f3c35
Small bugfix + refactor email reports (#3642)
* use more convenient testing functions

* do not display + sign with 0% change in emails

* Rename module/file/function names

before, `weekly_report` was also used for monthly reports and that was a
bit confusing to read in code.

* Refactor send_email_report.ex

This commit improves readability by refactoring the code into smaller
functions and reducing the number of arguments given to functions.

But more importantly, it stops making duplicate stats queries for every
email recipient by moving the queries out of the for loop.

* Refactor: move querying logic out of the worker module

and merge all stats information under a single `stats` assign.
3 years ago
hq1 b96319321c
Add Browser/Version breakdown to CSV (#3599)
* Add csv for browser versions

* add changelog

* Fixup - ensure the dashboard browser version breakdown still works

* Update CHANGELOG

---------

Co-authored-by: Ekaterina Krivich <ekaterinak@heathmont.net>
3 years ago
hq1 7bf1e2a6ed
Reapply "Define a better monthly pageview usage (#3564)" (#3574)
This reverts commit c739b8878d.
3 years ago
hq1 5278c23965
Revert and extract migration (#3573)
* Revert "Define a better monthly pageview usage (#3564)"

This reverts commit 57188a402a.

* Extract migration from 57188a402a/priv/repo/migrations/20231129103158_add_allow_next_upgrade_override_to_users.exs
3 years ago
RobertJoonas 57188a402a
Define a better monthly pageview usage (#3564)
* refactor asking for the monthly pageview usage

* add tests for usage and limits section in account settings

* display pageview usage per billing cycle for active subscribers

* disable cycle tabs if no usage

* make current billing cycle whole

...instead of capping it at today's date

* run queries for different cycles concurrently

* fix linebreak bug

* add calculate usage action into CRM

* change some names of assigns

* block subscribing to a plan by pageview usage

Depending on whether the customer has already subscribed or not, checking
their pageview usage is different:

* If they're not subscribed yet, we allow them to subscribe to a plan If
  it their last 30 days usage does not exceed the plan pageview limit by
  more than 15% (30% for when subscribing to a 10k plan)

* For existing subscribers, we'll use the exact same mechanism that we're
  using for locking sites - the last two billing cycles usage. If both
  cycles exceed the plan limit by more than 10% - we don't allow them to
  subscribe to the plan

* apply credo suggestion

* prevent highlight bar overflow

* move disabled classes to button element

* optimize for darkmode

* unify link and text styling on the same horizontal line

'Upgrade' & 'Update billing details' links + billing interval text were
positioned on the same line. The font size was similar, but not the same

* improve exceeded_limits function readability

* Refactor some tests and remove code duplication

* override allow upgrade when limits exceeded

In cases where limits are exceeded, we can set the boolean flag
`allow_next_upgrade_override` to `true` in the CRM. This will allow
the user to upgrade to any plan they want. After they've upgraded or
changed their plan - the flag will automatically reset to `false`.

* only apply upgrade override for exceeded pageview limit

* fix tests on the CI

* make current_cycle usage always displayed by default

* make pageview allowance margin more clear

* add comment
3 years ago
hq1 a4b9c3b8ba
Remove custom domains support + update build options (#3559)
* Disable super-admin checks on small build

* Mute a test writing to stdout

* Move sampling outside of small build

* Convert waiting_first_pageview to heex and stop relying on env vars

* Set site limit unlimited on small build

* Stop relying on app env to get trial expiry

* Remove custom domains - including migration

* Remove is_selfhosted from layout view

* Quota fixup

* Stop relying on app env for self hosted registration

* Stop relying on app env for pass reset success

* Apply on_trial? check only on full build

* Update templates relying on app env

* Adjusts auth controller tests for small build

* Trial fixup

* Fixup

* Stop relying on app env

* Rest of the fsckn owl

* Update typespecs

* Fix dialyzer warning

* Remove unused module

* Credo + format

* GeoIP is not, for full build

* Use `small_build?()` where applicable

* Implement bypassing FirstLaunchPlug without insertions

* Get Marko's patch de58a18a85

* Test is-dbip=false presence

* Fix typespec

* Remove future hardcodes

* Handle `nil` from `Plausible.Geo.database_type()`

* Remove XXX marker

* Use one typespec for two clauses

* Introduce `MIX_ENV=small_dev`

* Revert "Use one typespec for two clauses"

This reverts commit 8d8cd21764.
3 years ago
hq1 0f577a998c
Update credo config + extras (#3547)
* Disable CyclomaticComplexity checks

* Move AuhtorizeSitesAPI plug under extra
3 years ago
hq1 88e1d9dc28
Small build updates (#3546)
* Sites API

* Extract Revenue react api helpers

* !fixup

* Extract JS Money module to /extra

* Extract Revenue full build extras (tests pass for full)

* Update MIX_ENV=small mix test

* Remove dead code

* Add moduledocs

* Add credo config

* Trick dialyzer

* DRY revenue metrics

* Use more concise version of on_full_build macro

* Disable credo check
3 years ago
hq1 b9ec38038c
Add small build option (#3536)
* Update applications

* Clone community config

* Move modules to experimental dir

* Update runtime config

* Apply first set of compile-time conditionals

* Move funnel schemas to experimental

* Make funnel schema-less build compile

* Use experimental/lib for elixir code

* Move JS funnels to experimental

* Clean up conditional rendering

* Tidy up the pipeline

* Make two builds pass tests without warnings

* Reuse existing dotenvs

* Do a bunch of renames

* Clean up naming

* Run secondary CI

* Update router

* Remove RewriteFunnelDupes migration

Tests were disabled already and it was a one-off shot

* Fixup quota mixins

* Add moduledoc

* Change MIX_ENV for seconary test run

* Skip crm on small

* !fixup

* Exclude flags pipeline

* Update lib/plausible_web/controllers/stats_controller.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
3 years ago
Márton Salomváry 555eb25d20
Allow custom event timeseries in stats API (#3505)
* Allow custom event timeseries in stats API

* Fix linting error

---------

Co-authored-by: Uku Taht <Uku.taht@gmail.com>
3 years ago
ruslandoga 13055aafc0
use windows in 'time on page' queries (#3446)
* use windows in 'time on page' query

* add imported time on page back

* join imported_pages instead of extra query

* don't forget aggregated time on page!

* eh?

* no need for separate window clause

* use dynamic_filter_condition in aggregate_time_on_page

* it's avg, not total

* fix dynamic_filter_condition call

* tests pass

* fewer changes

* fewer changes + feature flag

* base quickfix

* fewer changes

* fewer changes?

* fewer changes!

* replace coalesce with if(empty(),etc)

* make window_aggregate_time_on_page return same result as neighbor_

* breakdown as well

* add failing test

* more info

* more info

* format example

* add failing aggregate.ex test

* fix breakdown.ex windowing time_on_page calculation

* fix aggregate.ex windowing time_on_page calculation
3 years ago
Vini Brasil c9bf5827e9
Prepare business tier for release (#3464)
* Change limits for trials

* Keep legacy trial limits for users that registered before the business tier

* Change private preview notice for release

* Run formatter

* Add countdown to private preview notice
3 years ago
Adrian Gruntkowski 07cab27fef
Implement new sites view (#3463)
* Implement complete basics of LV sites

* Reimplement everything in LV except pagination

* Implement basic search capability

* PoC: plot visitors on sites index

* Add rudimentary clipped gradient in minicharts

* Fix clipping gradient, define once

* Format

* Add moduledoc to visitors component

* Move paginator helpers to the top core namespace

* Fix typespec of `Plausible.Sites.list`

* Split sites component into subcomponents

* Add function to uniformly calculate 24h intervals
and visitor totals across multiple sites.

* Integrate batch 24h interval query with plots on sites view

* Don't confuse heex compiler with alpine @ shorthands

* Make linear gradient svg definition truly invisible

* Implement basic pagination

* Extract `site_stats` from site and invitation cards

* Improve pagination

* Tweak css

* Improve filtering on pagination and make WSS fail graceful

* Test `last_24h_visitors_hourly_intervals/2`

* Replace /sites with LV implementation

* Add debounce to search filter

* Fix typespecs

* Fix styling

* Fix mini graph scaling factor calculation

* Fix search consuming itself

* Minimal tweaks to the plots

* Fixup

* Remove magic numbers from the plot

* Create `site_pins` table

* Add `SitePin` schema

* Implement listing invitations, sites and pins in a single query

* Add FIXME note

* Remove site pins for now

* Add tests for `Plausible.Sites.list/3`

* Add a couple more tests to sites dead view

* Remove unnecessary FIXME

* Add LV tests for Sites

* Calculate and display 24h visitors change

* Render the change in bold

* Add clarfying comment on virtual field in `Site` schema

* Remove unnecessary function from Invitations API

* Remove unused list opt from type definition in `Sites`

* Improve joins in list query slightly

* Add comment on manually computing sites list total

* Start searching from a singly character in domain field

* Add typespec to `last_24h_visitors_hourly_intervals`

* Extend moduledoc in visitors component

* Simplify loading sites in LV

* Simplify assigns in LV

* Add missing group for shadow under site card

* Make invitation modal render

* Make HTML in sites LV semantically correct

* Remove autofocus and focus search on `/`

* Remove shadow from search input

* Make search cancel on escape

* Fix tests relying on outdated HTML structure

* Make visitor chart color scheme consistent with dashboard chart

* Update styling of trend labels

* Fix empty state and improve search blur/focus handling

* Use live navigation for pagination

* Implement spinner on load from search

* Remove unused `Plausible.Stats.Clickhouse.last_24h_visitors/1`

* Calculate uniques correctly across hour boundaries

* Swap inlined svg for Heroicons component in invitation modal

* Add order by to base query in 24h hourly intervals

* Revert "Add order by to base query in 24h hourly intervals"

This reverts commit a6be5e3026.

* Query clickhouse 24h visitors only on second mount

* Remove redundant sign from percentage change when negative

* Switch to offset-based pagination

  - offset seems easier to deal with for when actions on
    paginated list will be performed such as site pinning;
    tracking cursor data makes some entries disappear in
    edge cases. The data set is still fairly small and
    static, even for large customers.
  - we're removing Phoenix.Pagination as it doesn't really
    fir any use case, and it was only used to limit the number
    of sites in the site picker
  - site picker is now limited to 9 sites (future: pinned
    sites will be prioritized there)
  - no need to re-query for total count any more
  - BTW, the old /sites template was removed

* Refine the plot queries; Tests pass snapshot

* Add PromEx plugin for LiveView

* Fix tiny plot cut-off at the top

---------

Co-authored-by: Adam Rutkowski <hq@mtod.org>
3 years ago
Vini Brasil 896d78d8fd
Apply feature gates to dashboard queries (#3424)
* Read feature status from Billing.Feature instead of %Site{}

This commit changes data attributes passed to React. Previously the
controller read feature statuses directly from the %Site{} schema. The
Billing.Feature context is aware of the user plan and the features
available.

* Limit funnels internal API based on site owner plan

* Limit props internal API based on site owner plan

* Use site factory in QueryTest

* Limit custom property filter based on site owner plan

* Limit revenue goals queries based on site owner plan
3 years ago
Adrian Gruntkowski 27a11fc5b7
Filter out empty entries when listing stats for UTM props (#3339)
* Filter out empty entries when listing stats for UTM props

* Update test fixtures removing noref entries in UTM CSV stat exports

* Update external API tests to account for lack of noref records for UTM stats

* Filter out entries with empty UTM props from imported GA stats

* Remove unreachable GA utm_source dim clause in imported stats logic
3 years ago
RobertJoonas 5ba21c04fe
Remove Props FF (#3242)
* stop returning prop_names from /conversions

* remove the old prop-breakdown API route and CSV export

* remove allowed_event_props from CRM

* remove DeprecatedConversions

* remove props FF

* update changelog
3 years ago
Uku Taht ab4eb75109
Inflate sampled results in Stats.Funnel.funnel/3 (#3270)
* Inflate sampled results in Stats.Funnel.funnel/3

* Add test for sampling

* Update test/plausible/funnels_test.exs

Co-authored-by: hq1 <hq@mtod.org>

* Ignore unused binding

* Use bang version of hash function

---------

Co-authored-by: hq1 <hq@mtod.org>
3 years ago
hq1 e59cda5d93
Fix filter suggestions (#3265)
* Fix filter suggestions

* Mute credo for now
3 years ago
hq1 e467324c5a
Disallow small intervals on large ranges (#3245)
* Update interval dropdown based on custom range selection

* Add `monthsBetweenDates` function

* Reassign interval for custom "period" based on local storage

* Generate intervals by period based on stats_start_date

* Account for "custom" and "all" period intervals dynamically
3 years ago
Vini Brasil 8c3b541971
Allow internal prop keys in queries (#3231)
* Remove `Plausible.Sites.set_allowed_event_props/2` function

This commit removes the `Plausible.Sites.set_allowed_event_props/2`
function in favor of `Plausible.Props.allow/2`.

* Allow internal prop keys in queries

This commit allows internal prop keys by default in queries. I decided
not to include internal keys in the database, because they might change
and we'd need to migrate it again.

* remove redundant 'props.' from behaviours/index.js

---------

Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
3 years ago
RobertJoonas 7b39328d6c
Props details view (#3196)
* make (none) value in custom prop breakdown add +1 to pagination limit

This is needed because the (none) value is always added to the
breakdown_results **after** fetching those from ClickHouse.

* only include (none) values on the first page of results

* fix percentage metric calculation for paginated results

Instead of summing up the number of visitors from the breakdown results
to get the total, we have to make a separate query to `Stats.aggregate`.
Otherwise, the percentages for each results page will wrongly add up to 100%.

Since imported data for aggregated visitors and other properties (such as
browsers, OSs, etc) live in different tables, we have to tweak the tests to
also include the same number of visitors in the `imported_visitors` table.

* add details view for props

* changelog

* exclude imported data from total
3 years ago
Kris Buist 7c5ebab2c6
Fix the interval selection in the all time view (#3110)
Previously, the interval was always overwritten with the default value, which is determined based on the age of the site.
It should not be overwritten when the interval is explicitly passed by the user
3 years ago
RobertJoonas 51cad4c0ec
Props into CSV export (#3167)
* move set_allowed_event_props into context module

* extract parse_csv fn

* add custom_props.csv to CSV export (under FF)

* add test for custom prop export with a prop filter

* add test for goal filtered scenario

* mix format

* fix flaky test

* extract breakdown_custom_prop_values function

* Revert "fix flaky test"

This reverts commit 256b9bb2e7. The flaky
test was already fixed in 86557b1878

* make set_allowed_event_props a bang function

* test that all files are included in the CSV export
3 years ago
Vini Brasil 35107e2b8f
Add revenue metrics to prop breakdown (#3140)
* Extract <Money> React component

* Unhide prop breakdown for revenue goals

* Query revenue metrics on prop breakdown API
3 years ago